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1. Executive Summary 


Female students are much less likely than their male counterparts to take A levels in 
certain STEM (Science, Technology, Engineering and Maths) subjects. This is 
problematic for gender equality but also limits the country’s capacity to address the 
growing national STEM skills shortage. This research was commissioned to design and 
test the effectiveness of two sets of interventions based on behavioural insights theory, to 
increase the number of high-achieving female students choosing STEM subjects at A 
Level. 


The principle mechanism by which the interventions sought to increase uptake of STEM 
A Levels was by improving perceptions of STEM’s utility — this was the target of the 
parent-focused intervention, as well as two of the three student-focused intervention 
exercises. The secondary mechanism was designed to improve students’ self-concept 
and expectations of success in STEM. These interventions were based on insights from 
a rapid review of relevant academic research in the fields of psychology and behavioural 
science. 


Participating schools received either a student-focused intervention in isolation, or the 
student-focused intervention in combination with a parent-focused intervention. While 
schools were free to administer the interventions to any and all students approaching 
their A level choices, the interventions were primarily intended for girls who were likely to 
be eligible to take STEM subjects (referred to as ‘high-achieving girls’ below). 


The primary evaluation compared students’ responses before and after taking part in the 
interventions (Pre-Post study design). The key findings after the interventions, as 
compared to before, are: 


e High-achieving girls were more likely to state the intention to study two or more 
STEM A Levels. 


e This positive change was greater for high-achieving girls in schools that received 
the parent-focused intervention in addition to the student-focused intervention. 


e High-achieving girls were more confident that they could apply the material they 
learned in maths or science class to real life. 


e High-achieving girls were slightly less confident in their ability to learn the material 
they covered in maths or science and that they could be successful in these 
subjects 


e High-achieving girls were less likely to report having discussed the importance of 
maths or science with their parents in the previous month. 


e Students who received the parent-focused activities showed an increased 
likelihood of discussing A Level choices with their parent or guardian, relative to 
those who received the classroom-focused activities only. 


Following further evaluation of the data, the following additional findings were observed: 


e There was a strong relationship between the intentions of girls to study STEM 
subjects with their final A Level choices. 


e However, not all stated intentions to study STEM A Levels translated into formally- 
submitted A Level preferences (as was also the case for non-STEM A Levels). 


As the design of the intervention was changed ahead of the administration to schools 
(from a Randomised Control Trial (RCT) to a Pre-post study design), it is possible that 
factors besides the interventions themselves were responsible for the changes in 
students responses after participation compared to beforehand. Further research would 
be required before firm causal conclusions can be drawn. Given the positive indications 
of the interventions, especially in tandem, such further research is warranted. 


This report also seeks to make recommendations based on the efficacy of the 
interventions to inform further research and more broadly, to help develop targeted 
actions to improve diversity in STEM participation. 


The recommendations are: 


e That the observed benefits of the parent-focused activities in improving intentions 
of students to discuss A Level choices is studied in conjunction with existing 
careers guidance available in schools. 


e That further research using the parent-focused intervention is also explored, 
including broadening the type of communication parents receive as this places no 
burden on teaching time. 


e That capital-related inequalities and the action-intention gap are considered when 
carrying out further research. 


e That any changes presented by the impact of COVID-19 on STEM uptake 
(compared to other subjects) in the coming years are explored, with respect to 
how changes in teaching e.g. remotely or in smaller class sizes would have 
affected student cohorts in England. 


e That the focus shifts to the longer-term effects of societal norms surrounding 
STEM and perceptions about the challenges of the subject matter itself and 
interventions are targeted much earlier, particularly for girls. 


2. Background 


2.1. The Challenge 


The Social Market Foundation (2017) found that the profile of jobs in the UK is changing 
and roles which require Science, Technology, Engineering and Mathematics (STEM) 
skills will rise at twice the rate of other occupations between now and 2023, creating a 
particular demand for graduate level STEM skills (NAO, 2018). STEM graduates are 
already in short supply and the number of students taking STEM-related courses 
decreases as students progress through the educational pipeline. Furthermore, the 
challenges presented by the SARS-CoV-2' epidemic in conjunction with the already 
existing need to respond to the challenges of an increasingly technologically advanced 
world (BEIS, 2017), put STEM careers at the forefront of skills demands. 


The challenge to fill these posts will continue to grow unless more work is done to foster 
a pipeline of young people with STEM expertise. This will require understanding what 
works to encourage groups that are historically underrepresented to consider STEM, 
most obviously girls and women. For instance, Engineering UK (2019) found that only 
12.0% of workers in engineering occupations were female despite comprising 47.1% of 
the overall UK workforce in 2018. 


Despite outperforming boys in most GCSE STEM subjects?, girls are choosing other 
subjects over STEM at the first opportunity. A Levels (or alternative post-16 pathways) 
represent the first point at which students choose whether to proceed with STEM 
subjects, opening or closing opportunities to pursue STEM subjects in Higher Education 
or as a career path. At this point, gender disparity in the uptake of STEM subjects starts 
to emerge. In 2019, female students accounted for 44% of all STEM A Level? exam 
entries: representing just 13% of examination entries in computing, 22% in physics, and 
39% in mathematics‘. Importantly, relatively few girls opt for two or more STEM subjects 
at A Level (22% of girls vs. 35% of boys in 2019, with only 10% girls vs. 18% of boys 
taking 3+ STEM subjects®), which is a requirement to access many STEM degrees. 


In addition to contributing to the national STEM skills shortage, women’s 
underrepresentation in STEM fields also presents problems in terms of gender equality 
and diversity. As our society becomes increasingly dependent on technology, STEM jobs 
grow in terms of income, status, and influence as well as number; it is important that 
these jobs are not overwhelmingly held by a limited section of society. 


1 This research was undertaken prior to the epidemic. 

? In 2019, with the exception of physics - DfE, Key stage 4 performance 2019 (revised) (link) 

3 For the purpose of this research, STEM A levels were defined as Maths, Further Maths, Physics, 
Chemistry, Biology and Computer Science 

42019 Revised A level subjects by characteristics, DfE statistics: 2018-19 A Level results (revised) (link) 
5 2019 Revised maths and science tables, DfE statistics: 2018-19 A Level results (revised) (link) 
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Girls have the potential to access STEM subjects and these subjects open doors to 
careers that are in-demand and well remunerated®. While it has been found that 
comparative STEM advantage explains at least 12% of the under-representation of 
qualified girls in the earliest instance of STEM specialisation (Goulas, 2020), the gender 
gap in STEM A Level uptake cannot be fully explained by traditional economic theories. 
Behavioural science has the potential to complement traditional economics explainations 
and help us to further understanding the barriers to participation and identify appropriate 
solutions. 


2.2. Research Aims 


The target behaviours for the present research were: 


1. Increasing the number of girls choosing maths A Level 
2. Increasing the number of girls choosing two or more STEM A Levels 


The goal was to develop interventions, informed by behavioural insights, that could have 
an immediate impact on the number of girls who would go on to be eligible to apply to 
STEM degrees or career paths. As such, the present research sought to target the very 
next wave of students selecting their A Levels. 


The research aims also align with the wider DfE goal of increasing maths A Level take-up 
and policies such as the Advanced Maths Premium, as well as the Smith Review target 
of increasing the number of students studying maths until the age of 18. Girls are not 
underrepresented uniformly in all STEM subjects and indeed in 2019, made up 63% of 
biology and 54% of chemistry GCSE entries respectively. There are also a larger number 
of female (65%) than male (35%) graduates from Biological Sciences for instance at 
higher education’. While there are issues further up the chain in postgraduate research 
and indeed in employment, this research focuses on improving the pipeline at A Level. 


Additional requirements of the present research were that the interventions had a strong 
link to Careers Advice, Information and Guidance, be considerate of school workload, 
and be scalable, such that nationwide rollout would be feasible and affordable. 
Furthermore, the present research was to include a robust evaluation, such that the 
impact of the interventions could be accurately determined. 


® Higher Education Graduate Outcome Statistics: UK, 2017/18 (link) 
7 HE qualifications obtained by subject area and sex, HESA 2018/19 (link) 
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2.3. Rapid Evidence Review 


The Behavioural Insights Team conducted a rapid review of evidence relevant to the 
research aims, to investigate the factors contributing to the gender gaps in STEM A Level 
uptake and to identify interventions to encourage girls to consider STEM options. 


2.3.1. Why is there a gender difference in STEM A Level uptake? 


A number of academics have looked at this question through the lens of ‘expectancy- 
value’ theory (Eccles, 2019). According to this theory, the courses that a student selects 
depend on their ‘expectancy-value’ assessments of the available options. Expectancy 
refers to the student’s expectations of success. Value refers to the extent that the student 
values the option in terms of (a) its usefulness for fulfilling personal goals (‘utility value’); 
(b) the extent to which it matters for the student’s sense of self or identity (‘attainment 
value’); (Cc) how much the student actually enjoys it (‘intrinsic value’); and (d) the costs 
associated with the option, including social, psychological and economic costs, as well as 
the opportunity costs of alternatives (‘costs’). 


These expectancy-value motivational beliefs are influenced by the individual’s 
experiences, which will be affected by the culture in which the individual lives (Eccles, 
2015). In particular, gender-role socialisation will shape these beliefs. However, this does 
not necessarily translate across to all STEM subjects. Archer (2020) found that biology A 
Level students, are three times more likely to be female than physics students and one 
and a half times more likely to be female than chemistry students. There is an 
association of science with ‘masculinity and cleverness’ but this was particularly 
pronounced in the case of physics compared with chemistry and biology (Archer, 2020). 


2.3.1.1. Expectancy 


Girls’ expectations of success in STEM subjects relative to others appear to be lower 
than those of boys even though their performance is no worse (Wang, 2013). 


e Poor STEM self-concept 


Despite girls outperforming boys in most STEM subjects at GCSE§, they are less 
confident in their abilities in STEM (Wang, 2013). This is likely due to stereotypes about 
STEM and gender. STEM subjects (particularly some, such as physics) are seen as 
being only for people who are very ‘brainy’ (Archer, 2020), and internalised gender 
stereotypes about brilliance discourage girls from engaging in such activities; girls as 
young as six endorse the notion that genius is an exclusively male trait (Brian et al., 


8 DfE, Key stage 4 performance 2019 (revised) (link) 
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2017). There is also a specific stereotype that girls are less naturally talented in STEM 
than boys (Eccles, 2015). 


A factor that may contribute to poor STEM self-concept in some cases is stereotype 
threat; this is the threat experienced when you perceive that your behaviour may be 
viewed through the lens of a negative stereotype. The stress and anxiety it brings can 
make it harder to perform well (Miyake, 2010). 


Beliefs about the nature of intelligence are also important here; if people believe that 
STEM ability is innate and fixed, then current perceptions of low ability in STEM will 
necessarily mean low expectations of success in the future. 


e Cross-domain comparisons 


Girls’ expectancy-value beliefs about non-STEM options are also relevant. Where girls 
have higher expectations of success, or assessments of their own abilities, in non-STEM 
subjects (Skaalvik, 1990), this will make them more inclined to choose those options. 
Gender stereotypes will influence girls toward this viewpoint. Furthermore, there is 
evidence to suggest that it is more common for girls to demonstrate high verbal abilities 
as well as high maths abilities (Wang, 2013), meaning that, among students eligible for 
STEM A Levels, the girls are more likely than the boys to be drawn in other directions. 
There is also evidence, discussed in the section below, that perceived abilities in non- 
STEM subjects could negatively impact self-concept in STEM. 


e Parents and teachers 


Parents and teachers both influence girls’ belief in their ability to succeed in STEM 
subjects. Parents’ beliefs about their child’s abilities in a given subject predict those of the 
child, even when controlling for teachers’ assessments (Eccles, 2015). And, importantly, 
parents’ beliefs are influenced by the gender of their child; even when girls outperform 
boys in both maths and English, parents are more likely to think sons are talented in 
maths than daughters, and vice versa for English (ibid). 


Furthermore, there is evidence that the better a parent thinks their child is at English, the 
worse the child thinks they are at maths. Girls’ self-confidence in maths is more likely to 
suffer via this mechanism than boys’, because girls are more likely than boys to be 
thought of as being good at English by their parents (ibid). 


Similarly, teachers are more likely to attribute girls’ success in physics to ‘hard work’ and 
boys’ success in physics to being ‘naturally bright’, even when they do not perform as 
well as girls (Archer, 2013). There is also evidence that teachers stereotype maths as a 
male domain and consider girls to have lower ability in the subject than boys (ibid). 
Differential teacher expectations for achievement translate into differential treatment of 
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students. Importantly, the more a teacher stereotypes maths as a male domain, the more 
strongly their students endorse the maths gender stereotype (Keller, 2001). 


2.3.1.2. Value 


Girls’ perception of the personal value of STEM subjects relative to others is lower than 
that of boys (Eccles, 2015). 


e Personal goals 


A major factor that discourages students from choosing STEM is that they doubt the 
usefulness, or relevance, of STEM subjects for their lives (Archer, 2013). Girls are more 
likely than boys to endorse communal goals (e.g. working with or helping others) and 
there is a stereotype that STEM subjects do not help fulfil these goals (Diekman, 2011), 
with the exception of biological and other health sciences, in which females are not 
underrepresented. 


Most young people have a narrow view of the usefulness of STEM subjects and where 
they can lead; for many young people, science qualifications appear to primarily lead to 
being a scientist, a science teacher or a doctor. Therefore, if a young person does not 
intend to pursue these roles, they are more likely to see post-16 science subjects as ‘not 
relevant for me’ (Archer, 2013). 


e Social norms 


Humans are social creatures and, as such, our behaviour is heavily influenced by our 
perception of what other people are doing, especially people who we consider to be like 
ourselves; we tend to follow suit. There is a perception that STEM subjects are more ‘for 
boys’ (Girlguiding, 2017) or as Archer (2013) posits, girls tend not to see these subjects 
as ‘for them’. Girls do not see ‘people like them’ represented in STEM (Cassidy et al., 
2018) and report a relatively low sense of belonging in STEM fields. STEM subjects do 
not align well with the stereotypical female gender identity (Archer, 2013) and there may 
be high costs for girls in choosing to participate. 


e Cross-domain comparisons 


Value judgements about non-STEM subjects are also relevant here, as an individual can 
only take a limited number of A Levels. There is evidence that girls may value non-STEM 
subjects more than boys do. For example, in a German study, girls were found to value 
English as a subject more than boys (Lauerrmann et al., 2015; Eccles, 2015). Girls are 
more inclined than boys to be interested in careers related to ‘caring’, human-services 
and the arts (ibid) and therefore girls are more likely to value school subjects that they 
see as likely to help them achieve these goals. 
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e Parents and teachers 


Parents and teachers are key influencers of students’ beliefs about the value of STEM for 
them (Harackiewicz et al., 2012). Students are much more likely to select post-16 maths 
and/or physics if a key adult - typically a family member or teacher - has conveyed to 
them the worth of the subject, along with the belief that they can do well in it (Archer, 
2013). 


However, teachers report limited knowledge about the range of STEM careers and 
research finds that most parents have narrow views about where science can lead (ibid). 
Furthermore, given parents’ and teachers’ biases to consider STEM a male domain, 
these influencers are less likely to encourage girls into these subjects (Newall et al., 
2018). Indeed, recent DfE research (Ipsos Mori, 2019) found that parents/carers of boys 
are more likely to discuss careers in STEM with their child than parents/carers of girls 
(70% vs. 56%). Parents can share gender-stereotyped views of occupations with their 
children (Ikonen et al. 2017) and Tenenbaum and Leaper (2003) found that parents of 
daughters are less likely to believe that their child is interested in science. 


2.3.2. What works? 


Relatively little is currently known about what works to increase STEM course uptake as 
there have been very few rigorous evaluations of interventions with course choice as an 
outcome measure. 


2.3.2.1. Targeting students’ utility perceptions 


Nevertheless, a number of interventions have been shown to shift young people’s 
expectancy-value beliefs about STEM or self-reported motivation to participate in STEM. 
Many of these have targeted students’ perceived STEM utility. Some have provided 
students with information about STEM’s utility (Durik, 2007) and others have invited 
students to make their own arguments for how the STEM subjects they are studying are 
relevant for their lives (Hulleman, 2010). This ‘self-persuasion’ is thought to be 
particularly powerful because it is less didactic and avoids compromising young people’s 
autonomy, and because we are typically more readily persuaded by our own arguments, 
partly because they are tailored to our own interests and goals (Yeager et al., 2014). 
There is some evidence, published by Gaspard and colleagues (2015), that providing 
students with quotations from other students about the usefulness of STEM, and inviting 
them to evaluate the personal relevance of the quotations, might be the most effective 
strategy. This is likely because some students struggle with self-generating arguments 
about the personal usefulness of STEM. Furthermore, the other students, who provide 
the quotations, act as relatable messengers conveying the personal utility of STEM 
(Dolan, 2010). 
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Gaspard’s intervention also sought to target girls in particular. It included information on: 
the relevance of STEM for traditionally female, as well as male, activities and careers; the 
importance of effort and self-concept for success; and the unhelpful frame-of-reference 
effects that can occur in the classroom - this is when people derive their STEM self- 
concept by comparing their performance to those of others in the class rather than to 
their own performance over time. The study’s results showed some evidence of stronger 
effects on the perceived value of STEM for girls than boys. 


Another intervention that was specifically intended to appeal to girls was designed to 
show how the career of a scientist can actually afford communal goals (Diekman, 2011). 
Girls were more positive about the career of a scientist when it was presented as 
involving more collaborative as opposed to independent work. 


Similarly, an intervention designed to foster a ‘self-transcendent’ or prosocial purpose for 
learning has been shown to increase persistence with subjectively boring maths 
exercises and to increase attainment in STEM courses (Yeager et al., 2014). This 
intervention is referred to as a ‘purpose for learning’ exercise. It first primes prosocial 
thinking by asking students how the world could be a better place. It then uses statistics 
and testimonials from other students to create the social norm that ‘people like them’ are 
motivated to study in school because it will allow them to help others in the future. Finally, 
the intervention asks students to write a message to future students about their own 
reasons for studying; in addition to the benefits of self-persuasion described above, the 
act of advocating for a message helps people to internalise it. 


2.3.2.2. Targeting parents’ utility perceptions 


Judith Harackiewicz and her colleagues in the U.S. took a different approach: they 
developed an intervention that targeted perceptions of STEM’s utility in parents. They 
found this intervention to have a significant impact on STEM course uptake in the high- 
school-age students (aged 15-18) of the targeted parents (Harackiewicz, 2012). The 
parents were provided with information, via brochures and a website, about the relevance 
of STEM subjects for a wide range of educational options and careers as well as daily life 
activities. Importantly, parents were also given guidance on how to talk with their child 
about course choices and help them to make connections between STEM and the things 
they really care about in their lives. In a randomised controlled trial, this intervention was 
found to increase the science and mathematics courses taken by students in the last two 
years of high school by, on average, nearly one semester (ibid). The study also 
conducted surveys and found that the intervention promoted mothers’ perceptions of 
STEM’s utility value as well as conversations between parents and their children about 
course choices, which, in turn, both contributed to improved perceptions of STEM'’s utility 
value in the students. 
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2.3.2.3. Belonging 


There is some evidence that relatable role models may have a role to play in 
encouraging girls into STEM (Drury, 2011; Van Camp, 2019). However, it appears to be 
very important that role models not conform to STEM stereotypes (e.g. being geeky and 
socially awkward) (Cheryan, 2011; 2013). 


Another potential approach is the de-stereotyping of STEM environments; one study 
found that girls were more likely to express interest in studying computer science when 
shown a classroom that did not conform to STEM stereotypes compared to when they 
were shown a classroom that did (e.g. decorated with Star Wars posters etc.) (Master, 
2016). This effect was mediated by an improved sense of belonging. However, this would 
not have been a practical option for the current project. 


‘Social-belonging’ interventions (Walton, 2007; 2011; 2015) have been shown to increase 
academic course attainment amongst negatively stereotyped groups. These interventions 
use relatable messengers to create the social norm that everyone worries that they do 
not belong at first, but that this fades over time. The interventions also ask the students to 
contribute their own message for future students to help them settle in, thereby making 
them feel like the architects, rather than the recipients, of the intervention, and helping 
them to internalise the message. However, while these interventions have been shown to 
decrease course drop-out, there is little evidence for their efficacy in encouraging 
negatively stereotyped groups to opt in to potentially threatening environments. The 
same can be said for Values Affirmation exercises, which are designed to overcome 
stereotype threat by affirming one’s sense of self outside the realm of academic study 
(Miyake et al., 2010). 


2.3.2.4. Expectancy 


Despite calls to do so, relatively little attempt has been made to specifically target 
expectations of success in STEM, besides the intervention elements mentioned above 
(those designed to emphasise the importance of effort and self-concept and to 
discourage comparing oneself to others) (Gaspard et al., 2015). 


An intervention that normalises the need to put in effort to achieve success on university 
STEM courses has been found to increase motivation in women (Smith, 2012). This 
study did not measure expectations of success but did find an increase in sense of 
academic belonging. 


Forewarning activities have also been used to improve performance in STEM courses 
(Aronson, 2009). These activities inform students about the concept of stereotype threat, 
with the goal of helping girls attribute performance anxiety to this concept rather than 
their own abilities. However, as with the social belonging and values affirmation 
exercises, they are not typically used to increase course uptake, and their influence on 
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attainment (in which there is no gender gap in STEM GCSEs in any case) may only be 
effective in scenarios where the negatively stereotyped group is a numerical minority 
(O’Brien et al., 2019). 


2.3.2.5. Conclusion 


In conclusion, the evidence base particularly supports interventions that target 
perceptions of STEM’s utility for fulfilling students’ personal goals. Both interventions that 
target students directly, and those that target their parents, show promise. Girls are likely 
to be particularly receptive to interventions emphasising the ways in which STEM 
subjects can help achieve communal and altruistic goals. Relatable, and non- 
stereotypical, role models and messengers may also be helpful as well as attempts to 
increase girls’ self-confidence and expectations of success in STEM subjects, for 
example through the normalisation of effort. 
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3. Interventions 


The aim was to increase the number of girls choosing two or more STEM A Levels and to 
improve maths uptake. Based on the evidence, both a parent-focused and student- 
focused intervention were developed - each drawing on previous promising interventions 
in this space. The hypothesis was that by improving expectancy-value beliefs about 
STEM for girls and their parents the likelihood of girls choosing STEM subjects for A 
Level could be improved. 


3.1. Student-focused intervention 


A student-focused intervention was developed, which was a series of three exercises for 
students to do individually, on a computer, in a classroom setting. It was at each school’s 
discretion when these exercises were delivered over the intervention period (February to 
April 2019), providing there was at least a week between consecutive exercises. 


3.1.1. Expectancy exercise 


The first of the exercises was designed to boost students’ self-concept and expectations 
of success in STEM. 


The exercise began with a brief context-setting introduction, emphasising the increasing 
importance and communal value of STEM subjects, and outlining the content of the 
upcoming exercise as an opportunity to support fellow students to achieve their potential 
as they approach GCSEs and decide what to do next. 


It went on to explain that many students underestimate their abilities in STEM and that 
this holds them back from achieving their potential because self-belief is an important 
factor for success. It presented ‘three lessons for achieving success’: 


1. Effort is the most important factor for success and everybody can improve; 

2. Don’t judge yourself against other people; 

3. Track your progress in each subject over time and avoid comparing across 
subjects. 


The exercise then explained that messages from fellow students can help people to 
believe that they can do well in STEM. It presented an example message. Finally, the 
exercise invited the student to write a message to help younger students believe in their 
ability to do well in STEM. 
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3.1.2. Utility exercise 


The second exercise in the series, which was administered at least a week after the first, 
began with a brief recap of the first exercise. 


Next, the exercise explained why it matters if someone wrongly believes that they are 
bad at STEM subjects or assumes that these subjects are not ‘for them’; it makes them 
more likely to miss out on the benefits of continuing to study them after GCSEs. Then 
followed a summary of some of the key ways in which STEM A Levels are useful for 
university, jobs, making a difference and daily life. 


The remainder of the exercise was based heavily on the quotation intervention developed 
by Gaspard and colleagues (2015). Quotations from other students about the usefulness 
of STEM A Levels for their own lives were presented. The students completing the 
exercise were asked to: reflect on whether they had heard similar arguments before; 
identify which of the quotations they can most relate to and why; order the statements in 
terms of personal relevance; and contribute their own statement about why STEM 
subjects are important to their life - an opportunity to self-generate arguments. 


The exercise ended with a suggestion that the student talk to their parent/carer about the 
ways that STEM can be useful in life and the value of carrying on with these subjects 
after Year 11. 


3.1.3. Purpose for taking STEM exercise 


The third exercise, which was administered at least a week after the second one, was a 
version of Yeager’s ‘purpose for learning’ activity, geared towards STEM A Levels. 


The exercise started with a recap of some of the career opportunities to which STEM 
subjects can lead, including a number of examples that afford communal and altruistic 
goals. Next, survey statistics and testimonials from students about the reasons they 
chose to continue with STEM subjects at A Level were presented. These created the 
social norm that many students choose to study STEM at A Level because it will allow 
them to help people in the future. Finally, the students taking the exercise were invited to 
write their own testimonial (for future students) about how STEM A Levels could help 
them become the kind of person they want to be or make the kind of impact they want to 
have on the people around them or society in general. 


At the end of the exercise, students were, once again, reminded to talk with trusted 
adults, such as their parents, about their A Level choices. They were invited to give the 
decision careful consideration, as it will affect their options later on, rather than going with 
their gut instincts or doing the subjects their friends are doing. They were encouraged to 
write down a few ideas of the people they are going to talk with, or things they are going 
to find out about, to help with their decision about what to study. 
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3.2. Parent-focused intervention 


In collaboration with Judith Harackiewicz, a second, parent-focused intervention was 
developed, based on the one she used to increase STEM course uptake in the US, but 
adapted for the UK context. The intervention was a parent-facing website, which provided 
information about the usefulness of STEM A Levels for a wide range of educational 
options and careers as well as daily life. It also provided guidance for parents on how to 
talk with their child about A Level choices and how to support them to make connections 
between STEM subjects and their personal goals and interests. Notably, the website 
contained examples of how STEM subjects could help fulfil communal and altruistic 
goals, and hosted many images of students working together on STEM activities, 
including a large number of females who did not conform to stereotypes about the kind of 
people who work in STEM. One of the web pages also contained videos of students 
talking about the ways in which STEM A Levels had been relevant to their lives. 


Parents received emails signed by an authority figure at their child’s school, directing 
them towards the website. 


The original intention was that these messages would also communicate their daughter’s 
potential to do well in STEM subjects, thereby increasing expectations of success and 
perceptions of ability in STEM. However, following the trial design changes, the 
intervention was made available to all students not just high-achieving girls and the 
wording was adjusted accordingly. 


Please see the Annex for the detailed intervention materials. 
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3.3. Theory of Change 


INPUTS ACTIVITIES OUTPUTS OUTCOMES 


Year 11 female student 
applies to take maths 
A-level 


Year 11 female student 
applies to take multiple 
STEM A-levels 
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4. Research Design 


The final trial design was a pre-vs-post research design in which the student-focused 
intervention was made available to all students in the target year group at all participating 
schools, while half of participating schools were randomly allocated to also receive the 
parent-focused intervention. The primary outcome measure was students’ intended A 
Level subject choices, as measured in a survey at the beginning and end of the series of 
student-focused intervention exercises. This was a shift from the initial design, which led 
to some implementation and evaluation challenges, described below. 


4.1. Design revision 


The original intention for this research was that the evaluation would take the form of an 
individual-level randomised controlled trial (a commonly-used trial design, including in the 
Behavioural Insights Team’s previous work with the DfE°) with students’ A Level subject 
choices as the primary outcome measure. In this design, once schools were recruited to 
the project, the girls who were soon to make their A Level decisions and were likely to be 
eligible for STEM subjects (based on predicted grades and schools’ requirements - 
please see the ‘High-achieving Girls’ section of the Annex) would be randomly allocated 
to one of three conditions: 1) parent-focused intervention; 2) parent-focused intervention 
+ student-focused intervention; or 3) control. During the classroom-based sessions in 
which girls in condition 2 completed the student-focused intervention exercises, girls in 
conditions 1 and 3, as well as the other students in the classroom, would complete 
control versions (designed to be visually virtually identical to the intervention exercises, 
but to concern physical health and screen time). 


However, to ensure that provision did not vary within schools, a decision was taken not to 
proceed with an individual-level randomisation. Students/parents talk about what they are 
doing, which might have led to some sharing of the information and cross-contamination 
across groups. It was felt that excluding some students within a school from the 
intervention may have negatively impacted their expectation of success in STEM and it 
was important this was avoided. This decision meant that the evaluation strategy needed 
to be revised, but the timing of the decision limited the options for alternative trial 
designs; schools were expecting to receive interventions for their students and, indeed, 
had already scheduled the computer-based exercises, due to start within a matter of 
days of the decision. This meant it was not possible to have ‘pure’ control schools or to 
implement a stepped wedge design. 


° DfE (2018) Improving engagement and attainment in maths and English courses: insights from 
behavioural research 
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Instead, the control student exercises were removed from the design and the DfE 
permitted schools to extend the student intervention exercises to all students in the year 
group, male and female, as they saw fit. 


4.1.1. Primary evaluation: pre-vs-post 


The primary evaluation was changed to a pre-vs-post research design. Survey measures 
were added to the beginning and end of the series of student-focused intervention 
exercises. The primary outcome measure became intended A Level subject choice, with 
expectancy-value motivational beliefs about STEM as secondary outcome measures. 
The measures collected after the last exercise were compared to those collected at the 
beginning of the first exercise to provide an indication of the effect of the interventions. 


4.1.1.1. Comparing the results for different treatments 


All students involved in the trial received the student-focused intervention. Additionally, 
the parent-focused intervention was made available to just half of the participating 
schools for the trial period, randomly allocated (please see the Annex for the 
randomisation balance checks). This made it possible to compare these two 
intervention conditions (student-focused intervention only vs. student-focused 
intervention + parent-focused intervention). 


However, it should be noted that this comparison was not properly powered because the 
power calculations that determined the target sample size were based on an individual- 
level (rather than school-level) randomised design. This means that null effects were 
likely for these analyses even if the parent-focused intervention did add value over and 
above the student-focused intervention. 


4.1.1.2. Implications of design revisions 


The main consequence of moving from an individual-level randomised controlled trial to a 
pre-vs-post evaluation is that stronger assumptions are needed to interpret the results as 
causal impact of the intervention on the outcome. This is because, in a pre-post 
analysis, the behaviour of girls observed before the intervention act as a counterfactual 
(or control) to the behaviour of the same girls observed after the intervention. This 
counterfactual suffers some limitation, as other factors may have changed before and 
after the intervention and may be confounded with it. For example, if students’ stated 
intentions to take STEM A Levels had increased over the intervention time period, it is 
not certain that this would not have happened anyway. 


Similarly, if stated intentions to take STEM A Levels had declined over the intervention 
period, it could have been that the interventions actually counteracted a steeper decline 
that would have happened in the absence of intervention. It is, in fact, quite easy to 
imagine that stated intentions to study STEM A Levels would decline over time: as 
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students get closer to the actual decision point, they find out which subjects their friends 
are taking and they get advice from others e.g. about which subjects are relatively difficult 
at A Level. The causal interpretation of the results relies on assuming that no such 
changes would have happened, absent the intervention. 


4.1.1.3. Comparing the results for girls and boys 


Another implication of opening the interventions up to all students, male and female, 
rather than targeting them at high-achieving girls, was that this may have limited the 
ultimate potential effectiveness of the interventions. Interventions could no longer 
reference students’ particular potential in STEM subjects, as a means of boosting 
expectations of success. Furthermore, sixth forms may have a maximum capacity for 
their A Level classes. If the interventions encouraged boys to take STEM A Levels (and 
there is evidence (Harackiewicz, 2012) that the parent-focused intervention might be 
effective for lower-attaining boys but not lower-attaining girls) then this would have limited 
the number of slots available to more girls. 


As the interventions were made available to both genders, whether girls and boys 
responded differently was also examined. 


4.1.1.4. Correlation between the intention to study STEM subjects and actual A 
Level choices 


A correlation analysis was run to establish the extent to which students’ intention to study 
STEM A Levels, as stated in the ‘post’ survey, predicted students’ actual STEM A Level 
choices, as submitted to their school. 


4.1.2. Secondary evaluation: cross-cohort 


As a secondary evaluation, the A Level subject choices of the year group involved in the 
trial (the ‘treated cohort’) and the previous year group (‘untreated cohort’) within schools 
that had sixth forms were compared with one another. An increasing proportion of girls 
taking STEM A Levels could provide indicative evidence of the effect of the interventions. 
However, once again, as with the primary pre-vs-post evaluation, causal inferences is 
highly caveated for this analysis. It should also be noted that (a) this evaluation was not 
restricted to the high-achieving girls that were the intended recipients of these 
interventions and (b) the extent of the current year group that was treated was left up to 
the individual schools to decide. 


Please see the Annex for more information on assumptions and considerations. 
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4.2. Sample and eligibility 


The sample was all students about to select their A Levels (i.e., students in either year 10 
or 11), and their parents, at a sample of state-funded mainstream English secondary 
schools. 


While not the only recipients of the interventions, the analyses focused on a subgroup of 
high-achieving female students in these schools: these were the girls who were eligible 
for at least one STEM A Level subject according to their predicted GCSE grades and the 
school’s requirements, where provided - please see the ‘High-achieving Girls’ 
discussion within the Analytical Methods section. 


A subset of analyses focused on ‘very high-achieving girls’, defined, for the purposes of 
this project, as those predicted GCSE grades of 8 or higher in all their maths and science 
subjects. 


The sample for the secondary (cross-cohort) analyses were participating schools with 
sixth forms. 


The intervention trial period for this project was pre-specified as spring term 2019. 
Therefore, schools whose year 11 students had already submitted their A Level subject 
preferences prior to this period worked with year 10 students instead of year 11. This 
represented around 80% of the participating schools. 


4.2.1. Sample size 


89 schools were recruited to the study and returned a Memorandum of Understanding 
(MOU). Of these, 55 schools provided survey data on the intended A Level subject 
choices of their students before and after the exercises. This provided 3,872 students 
with a complete set of data on primary outcomes, 1,625 of which were high-achieving 
girls (those likely to be eligible to take at least one STEM A Level). 


This represents a high level of attrition. Much of the attrition was related to schools’ 
concerns around GDPR legislation. The 55 schools used in this analysis had an average 
pass rate (grade 5 or higher) in both English and Maths of 52%. This compares to 48% 
amongst the other 34 schools who originally signed up to the trial and 43% of all other 
mainstream state funded schools nationally. 


47 schools provided data of the final A Level subject choices of their existing cohort, 
either through manual returns or via the online portal provided. Of those students that 
had a complete set of data on primary outcomes, 2,846 students’ A Level choices were 
shared, 1,318 of which were high-achieving girls. 
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30 schools with sixth forms provided complete summaries of the A Level choices of their 
previous cohort too. These were used for school level modelling. 


Please see the Limitations (section 7.2) for attrition-related discussions. 


4.3. Outcome measures 


4.3.1. Primary evaluation: pre-vs-post 


Survey measures were collected at the beginning and end of the series of student- 
focused intervention exercises. '° 


The primary outcome measures for this evaluation were high-achieving girls’ stated 
intention to study A Levels in: 

e Maths 

e One or more STEM subjects at A Level 

e Two or more STEM subjects at A Level 

e Physics 

e Computer Science 
(the latter two being the subjects in which girls are most underrepresented). 
The secondary outcome measures for this evaluation were high-achieving girls’ self- 
reported: 

e Expectancy of success in STEM 

e Perceived utility of STEM 

e Discussion with parents about A Level choices 

e Discussion with parents about the value of STEM subjects 


Please see the Annex for these survey measures. 


4.3.2. Secondary evaluation: cross-cohort 


Students’ A Level subject choices were shared by schools (for both the current and 
previous cohorts of students, in the case of schools with sixth forms). In some cases, 
schools chose to have their students complete a secure online survey of their A Level 


1° In case a large proportion of students did not make it to the end of the third exercise, A level subject 
choice intentions were also surveyed at the beginning of the third exercise. However, completion rates 
were acceptable at therefore the data from the end of the exercise was used for analysis. 
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choices instead. For the cross-cohort evaluations, the outcome measures were the 
proportion of girls (enrolled in the school for both year 11 and year 12) who had chosen 


to study A Levels in: 


e Maths 
e Physics 


e Computer Science 


e One or more STEM subjects 


e Two or more STEM subjects 


4.4. Trial procedure 


Table 1 Timing of intervention delivery 


Timing 


Activity 


Autumn 2018 


Schools recruited. 


January/February 
2019 


Schools provided baseline data required for identification of high- 
achieving girls. 
Schools were randomly allocated to one of the two treatment 


groups: student-focused intervention only or student-focused 
intervention + parent-focused intervention. 


February - April 
2019 


All schools administered the student-focused intervention 
exercises to as much of the year group as they saw fit. 


Survey measures for the primary evaluation were collected at the 
beginning and end of the series of student-focused intervention 
exercises. 


Emails directing parents to the parent-focused intervention 
website were sent to parents of students in half of the schools. 


Summer 2019 - 
February 2020 


Schools provided A Level subject choice data for the secondary 
evaluation 
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5. Analytical Methods 


The survey responses of students pre- and post-intervention were collected through a 
bespoke online platform and were joined with the baseline data provided by schools on 
the attainment and characteristics of students participating. While the interventions were 
open to all students, this analysis focused on the survey response of ‘high-achieving’ girls 
only (as defined in the annex). 


In addition, all results were repeated on a subset of this group including only those girls 
receiving a grade 8 or higher in their Maths and Science subjects. This group is referred 
to as ‘very-high achieving’ girls in this analysis. 


5.1 Calculation of the overall treatment effect 


The overall treatment effect of high-achieving girls was determined using a pre-vs-post 
evaluation strategy. A series of student-level linear probability models was produced to 
test how the primary and secondary outcomes recorded differed before the first student 
exercise and after the third student exercise. 


For each model, only students who provided a response both before and after the 
intervention were included. A dummy variable ‘After Treatment’ was created to capture 
whether a student’s response was recorded pre-intervention (value of 0) or post- 
intervention (value of 1). Student level-fixed effects were applied to each of the models. 


Including student fixed effects helps control for unobservable factors which are student 
specific and would normally remain constant across the time period of this study. 
Examples include, a student’s work rate, overall enthusiasm for academic study and their 
teachers’ style and skills. These comparisons of outcomes over time, within the same 
student, are only possible for students surveyed before exercise 1 and then again after 
exercise 3; this reduces the overall sample size of each model. 


5.1.1. Primary outcomes 


The primary outcomes of this analysis are high-achieving girls’ stated intention to study 
STEM A Levels. Each of the five primary outcomes were converted to dummy variables, 
given a value of 1 if a student stated an intention to enter a subject(s) and 0 if they did 
not. 


In this setup, a positive value of the ‘After Treatment’ coefficient represents an increased 
likelihood in high-achieving girls’ intention to study that subject at the end of the trial. 
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5.1.2. Secondary outcomes 


The survey measures for a student’s ‘expectation of success in STEM’ and ‘perceived 
utility’ were based around their stated confidence in the following four statements: 


e | know! can learn the material we cover in maths or science 
e | believe | can be successful in maths or science 

e | can apply what | learn in maths or science class to real life 
e | think maths or science will be important to my future 


Students were given the choice of one of six options, which were then converted to a 
numerical scale ranging from 1 to 6, detailed below: 


e Strongly disagree =1 
e Disagree =2 

e Slightly disagree =3 
e Slightly agree =4 

e Agree =5 

e Strongly agree =6 


The regression models produced assume there is a linear relationship between the post- 
intervention dummy variable and the numerical scale of each outcome response. A 
positive value of the ‘After Treatment’ coefficient represents increased confidence in the 
statement post-intervention. 


The final two survey questions aim to understand the impact of the interventions on 
parental conversations. These were: 


e | have had at least one conversation with a parent or guardian about my A Level 
choices in the last month 

e | have talked to a parent or guardian at least once about the importance of maths 
or science in the last month 


The three possible responses, ‘Yes’, No’ and ‘Not Sure’, were converted into a binary 
outcome for each regression. A value of 1 was given for ‘Yes’ and a value of 0 for ‘No’ or 
‘Not Sure’. 


5.2 Comparing different treatments 


The relative impacts of the two treatment groups were compared for all primary and 
secondary outcomes. As before, individual-level linear probability models were set up for 
each of the outcomes, with an additional dummy variable ‘Parent-Focused intervention’ 
now added. Schools who received only the student-focused intervention were given a 


29 


value of 0, while those receiving the parent-focused intervention as well as the student- 
focused intervention were given a value of 1. 


The decision to randomly assign treatment groups at a school level rather than a student 
level has increased the likelihood that students’ responses to survey questions before the 
interventions will not be equally balanced between each group. The modelling follows a 
difference-in-differences design and aims to identify the relative change between each 
group. 


To achieve this, an interactive term was included in each of the models, ‘After treatment * 
parent-focused intervention’. A positive value for this coefficient highlights where the 
group who additionally received the parent-focused intervention have made a greater 
relative increase compared to the student-focused intervention only treatment group. 


5.3 Comparing the results for girls and boys 


After changes made to the experimental setup, boys and girls were both able to engage 
with the interventions designed. This allows us to test whether girls have responded any 
differently than boys have. Most schools provided information on the predicted grades of 
their boys as well as their girls; this allows us to restrict the sample to only high-achieving 
students who completed information on their intended A Level choices before and after 
receiving the interventions. 


As before, individual-level linear probability models were set up for each of the outcomes, 
with a new variable introduced to denote the gender of each participant. The interactive 
term ‘After treatment * Boys’ measures the differential effect between the genders over 
time. A positive coefficient would highlight a greater increase in intended uptake post- 
intervention for boys when compared to girls. 


5.4 Correlation between the intention to study STEM subjects 
and actual A Level choices 


The primary outcome measures, girls’ intention to study STEM A Level subjects, are 
based on the self-reporting of high-achieving girls in the pre- and post-intervention 
surveys. Schools were later asked to provide data on students’ formally submitted 
preferences, once these became available. This allows us to test how closely correlated 
a student’s intention to take a STEM A Level is with their formal choices. 


A series of linear regression models were produced for all high-achieving girls who 
provided survey responses and final A Level choices. A student’s chosen STEM A Level 
was used as the dependent variable and their stated intention to study that same subject 
was introduced as an independent dummy variable in the model. Other characteristic 
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information was also controlled for in the model, including the year group of the student, 
their ethnicity and whether they had ever been eligible for free school meals (FSM). 


5.5 Comparison of the treated year group with the previous, 
untreated, year group 


Schools with sixth forms were asked to provide summaries of A Level choices of their 
previous cohort split by gender. This group, who did not receive any of the interventions, 
can be compared to those involved in the trial. Due to the way schools reported this data 
the analysis focuses on school-level comparisons and focuses on all girls rather than 
high-achieving girls only. 


Our primary outcome variables were set as the percentage uptake of STEM A Level 
subjects at a school level. Linear regression models were produced using the variable 
‘Treated cohort’ to identify difference between the two groups. School-level fixed effects 
were included, and the academic year of the cohort and school type were also controlled 
for. Only schools that supplied a complete set of data for both cohorts were included in 
this model; this reduced the sample to only 30 schools. 


5.6 Assumptions 


5.6.1. Comparing different treatments 


Comparisons of the relative changes in outcomes of the two treatment groups are made 
using difference-in-difference modelling. Time difference in survey administration are 
used to capture the relative difference between each group over time. 


This effect has causal interpretation only under the assumption that absent treatment, the 
difference between the two treatment groups (if any) is constant over time. Despite 
having randomised schools to receive either student only or student + parent 
intervention, there were some significant differences in high-achieving girls’ survey 
responses at the start of the study. If, absent the intervention, the views of these girls 
naturally converged or further diverged over time, this change would be confounded with 
the relative effect of one intervention over the other. For this reason, results from this 
analysis should also be interpreted with caution. 


5.6.2. Cross-cohort 


In order to capture the effect of the treatment on girls’ actual A Level subject choices, we 
compared the choice of girls in the current cohort to the choice of girls in the previous 
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cohort in the same school. The effect measured in the secondary analysis (school level) 
has causal interpretation only under the assumption that: 


e the within-school-composition of Year 10 and Year 11 students is constant across 
cohorts in terms of all the characteristics the researcher cannot observe (ability, 
maths skills, motivation, etc.), 


e schools did not make any changes to the curriculum, the teaching staff, or in the 
teaching methods such that they could be confounded with the treatment, 


e the effect of any external factors (media, public opinion) in students’ A Level 
choices is the same for previous students and current students. 


As such, the estimation of the effect should be interpreted with caution. 


5.6.3. School-level randomisation balance checks 


70 schools agreed to participate in the trial. Schools were successfully randomised to 
parent and student vs student only interventions (Refer to Annex A). 


5.6.4. High-Achieving girls 


Girls who were on track to attain the prerequisite GCSE grades to be eligible for at least 
one STEM A Level subject from Maths, Further Maths, Physics, Biology and Chemistry 
were identified using the following method: 


e Schools that sent students’ predicted grades and A Level entry requirements 
(schools with sixth forms): For these schools, identification was straightforward. 
Whenever a predicted grade was accompanied with a +/- sign, it was rounded to 
the next integer (i.e., 5- and 5+ became respectively 5 and 6). 


e Schools that did not send students’ predicted grades but sent A Level entry 
requirements (schools with sixth forms): Using information from the sample of 
students with predicted GCSE grades, students’ predicted GCSE grades were 
estimated based on their KS2 Maths and English scores, year group, ethnicity, 
gender and FSM status. 


e Schools with sixth forms that did not send their A Level entry requirements: 
The mode requirement across the sampled schools for each subject was used. 
(Table 2) 
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Table 2 Mode requirement of sixth form schools 


A Level subject Mode requirement 
Maths 6 
Further Maths 7 
Biology 6;6 if double or 6 if single 
Chemistry 6;6 if double or 6 if single 
Physics 6;6 if double or 6 if single 


e Schools without sixth forms: 


The minimum requirement across the sampled schools for each subject was used. 
(Table 3) 


Table 3 Minimum requirement of schools without a sixth form 


A Level subject Minimum requirement 
Maths ts) 
Further Maths 6 
Biology 5;5 if double or 5 if single 
Chemistry 5;5 if double or 5 if single 
Physics 5;5 if double or 5 if single 
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6. Results 


6.1 Descriptive data 


6.1.1. Student characteristics 


Figure 1 shows the demographics of the 1,625 high-achieving girls included in the 
analysis of primary outcomes. It shows: 


e White students made up 71% of students within the sample; 14% were Asian, 6% 
were black, 6% were mixed and 1% other. 

e In comparison nationally'’ 75% of students were white, 11% were Asian, 6% were 
black, 5% were mixed and 2% other. 

e 12% of students were eligible for FSM within the sample compared to 27% of all 
students nationally. 


Figure 1 Characteristics of high-achieving girls sampled 


Ethnicity Ever Eligible for Free School Meals 


g White 


a No 
@ Asian/Asian British m Yes 
= Mixed/Multiple ethnic groups 
= Black/ African/Caribbean/Black 


British 
Other ethnic group 


‘1 Based on pupils at the end of KS4 in 2019, attending state funded schools 
https://www.gov.uk/government/statistics/key-stage-4-performance-2019-revised 
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6.1.2. School characteristics 


Figure 2 shows the school characteristics of the 1,625 high-achieving girls included in the 


analysis of primary outcomes. It shows: 


e There was a good regional distribution of participating schools, with the North 


West, East and Outer London the most represented. 


e 70% of students attended Academy Converter schools, 16% Community schools, 


9% Voluntary aided schools and 4% Academy sponsored schools. 
e 86% of students attended Year 10, with only 14% attending year 11. 


Figure 2 Characteristics of schools sampled 


Region School Type Year Group 


4% 4% 
8% 


a 


8% 


86% 


= North West 
m East 
= Outer London m Year 11 


m@ Academy Converter @ Year 10 
= Community School 
g Voluntary Aided School 


ae ne = Academy Sponsor Led 


= South East ; 

Inner London = City Technology College 
= North East Free School 
= South West Foundation School 


6.1.3. A Level Choices 


Figure 3 details the stated A Level choices of high-achieving girls when first surveyed 


prior to the first exercise. It shows: 


e 72% of high-achieving girls intended to take at least one STEM A Level and 44% 


intended to take at least two. 


e 40% intended to enter an A Level in Maths, 15% in Physics and 7% in Computer 


Science. 
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Figure 3 The stated intention of high-achieving girls to study STEM A Levels before the 


intervention 
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Figure 4 shows the responses of high-achieving girls to the attitudinal questions, when 
surveyed prior to the first exercise. It shows: 


e 52% of girls agreed or strongly agreed that maths or science would be important 
to their future. 

e 33% of girls agreed or strongly agreed that they could apply what they learned in 
maths or science class to real life. 

e 55% of girls agreed or strongly agreed they could be successful in maths or 
science. 

e 65% of girls agreed or strongly agreed they could learn the material covered in 
maths or science. 
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Figure 4 Response to attitudinal survey questions before the intervention 
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The responses of high-achieving girls’ to the parental-focused questions before the start 
of the trial were: 


e 47% of girls reported at least one conversation with parents about A Level choices 
in the month before the interventions were run. 

e 39% of girls reported at least one conversation with parents about the importance 
of maths and science in the month before the interventions were run. 
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6.2. Summary of outcome measures 


Summary of the outcomes of the intervention 


After treatment, there were the following significant changes for high-achieving girls: 


An increase in the stated intention to study at least two STEM subjects (2 ppt, p 
< 0.05) 

A increase in agreement with the statement ‘I can apply what | learn in maths or 
science class to real life’ (a 0.17 point increase in the 1 to 6 agreement scale, p 
< 0.001) 

A decrease in agreement with the statement ‘I Know | can learn the material we 
cover in maths or science’ (a 0.06 point decrease in the 1 to 6 agreement scale, 
p < 0.01) 

A decrease in agreement with the statement ‘I believe | can be successful in 
maths or science’ (a 0.05 point decrease in the 1 to 6 agreement scale, p $ 
0.05) 

A decrease in discussion of the importance of maths and science with 
parents/guardians (5 ppt, p < 0.001) 


For those who received the parent-focused intervention as well, there were the 
following significant changes: 


A greater increase in the stated intention to study at least two STEM subjects (4 
ppt, p < 0.05) 

A more positive change in discussion of A Level choices with parents/guardians 
(7 ppt, p < 0.01) 


6.3. Primary outcome measures 


The primary outcomes measured were based on girls’ stated intentions to study STEM A 
Level subjects and were based on survey data collected at the beginning and end of the 


trial. 


6.3.1. Overall treatment effect 


Table 4 shows that there was a statistically significant increase in high-achieving girls’ 
intention to study two or more STEM subjects at A Level after treatment. 


The positive coefficient of 0.02 is equivalent to a 2ppt increase in the likelihood of a girl 
taking two or more STEM A Levels after the treatment compared to beforehand: the 
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proportion of girls indicating that they would choose two or more STEM A Levels 
increased by about 5% (from 44% to 46%). 


As shown in Table 4, no significant changes were found in intention to study the 
individual STEM subjects (maths, physics, computer science or at least one STEM A 
level). It is possible that the intervention increased the probability of studying an 
additional STEM subject, on top of a STEM subject already selected. If the additional 
STEM subject is often different among girls (for some it is Maths, for some Physics, etc), 
then we may not be powered to detect the effect on each of these individual subjects, 
while we detect an ‘aggregate’ effect across STEM subjects’. 


Annex Table 1 shows the same analysis for the subset of 253 ‘very high-achieving’ girls 
(those with a predicted grade 8 in all maths and science GCSE subjects). No significant 
changes were found for this smaller group. 


Table 4 Overall treatment effect on intentions to study STEM subjects 


Maths Physics Computer 1+ STEM 2+ STEM 
Coefficient Name (Coef.) Sig. (Coef) Sig. Science Sig. | subject | Sig. subject | Sig. 
; ; (Coef.) (Coef.) (Coef.) 
After treatment 0.01 0.00 0.01 0.00 0.02 i 
(0.01) (0.01) (0.00) (0.01) (0.01) 
Student fixed 
effects yes yes yes yes yes 
R2 0.87 0.86 0.87 0.87 0.87 
Mean in pre- 
treatment 0.40 0.15 0.07 0.72 0.44 
N=1,625 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
1,625 high-achieving girls with complete data (those who provided this information both before exercise 1 
and then again after exercise 3). Standard errors are reported in parenthesis. * p < 0.05 **p < 0.01 ***ps 


0.001 


6.3.2. Comparing different treatments 


Table 5 shows that the increase after treatment — in high-achieving girls’ intention to 
study two or more STEM subjects at A Level — was significantly greater for the schools 
that received the parent-focused intervention as well (compared to those that received 


the student-focused intervention alone). 


As shown in Table 5, there were no significant changes for the individual STEM subjects. 


Annex Table 2 shows the same analysis for ‘very high-achieving’ girls only. For this 
smaller subset, there were no significant differences between the groups in the changes 
after treatment. 


It is worth noting that, despite the fact that schools were randomly assigned to the 
treatment groups, there were some differences in students’ stated intentions to study 


STEM subjects prior to treatment (indicated by the significant coefficients in the ‘parent- 


focused intervention’ rows of Table 5 and Annex Table 2). 


Table 5 Relative impact of different treatments on intentions to study STEM subjects 


1+ 2+ 
: Computer 
ee Maths : Physics : F : STEM : STEM : 
Coefficient Name (Coef.) Sig. (Coef.) Sig. hah Sig. subject Sig. subject Sig. 
; (Coef.) (Coef.) 
After treatment 0.00 0.00 0.01 -0.01 0.00 
(0.01) (0.01) (0.01) (0.01) (0.01) 
Parenttocused’. "| 951, |. 4 0.00 0.00 0.01 052 | * 
intervention 
(0.25) (0.19) (0.13) (0.23) (0.25) 
After treatment * 
Parent-focused 0.01 -0.01 -0.01 0.02 0.04 i 
intervention 
(0.02) (0.01) (0.01) (0.02) (0.02) 
Student fixed 
effects yes yes yes yes yes 
R2 0.87 0.86 0.87 0.87 0.87 
Mean in pre- 
treatment 0.40 0.15 0.07 0.72 0.44 
N=1,625 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
1,625 high-achieving girls with complete data (those who provided this information both before exercise 1 
and then again after exercise 3). Standard errors are reported in parenthesis. * p < 0.05 ** p< 0.01 ***ps 
0.001 


6.4 Secondary outcome measures 


Our focus now turns to analysing the secondary outcome measures. Students were 
asked six questions both before and after the intervention to identify changes to the 
following: 


e Expectations of success in STEM 

e Perceived utility of STEM 

e Discussion with parents about A Level choices 

e Discussion with parents about the value of STEM subjects 


The responses to these questions have been used to explore the overall treatment effect 
and the relative treatment effect of each intervention. 
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6.4.1. Overall treatment effect 


The outcomes of table 6 and 7 are based on the extent to which students agreed with 
statements about STEM, ranging from 1 (strongly disagree) to 6 (strongly agree). 


Table 6 shows that, after treatment, there were the following changes for high-achieving 


girls: 


An increase in agreement with the statement ‘I can apply what | learn in maths or 
science class to real life’ (a 0.17 point increase in the 1 to 6 agreement scale, from 
3.90 to 4.07) 

No significant change in agreement with ‘I think maths or science will be important 
to my future’. 

A decrease in agreement with the statement ‘I know | can learn the material we 
cover in maths or science’ (a 0.06 point decrease in the 1 to 6 agreement scale, 
from 4.66 to 4.60) 

A decrease in agreement with the statement ‘I believe | can be successful in 
maths or science’ (a 0.05 point decrease in the 1 to 6 agreement scale, from 4.39 
to 4.34) 


The mean in pre-treatment figures show the average response in the first survey. For 
example, a value of 4.66 for ‘I know | can learn the material we cover in maths or 
science’ indicates agreement levels sat between ‘Slightly Agree’ (a value of 4) and 
‘Agree’ a value of 5. An ‘After Treatment’ coefficient value of -0.06 indicates that the 
average response changed from 4.66 to 4.60. 
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Table 6 Overall treatment effect on expectations of success and the perceived utility of 
STEM subjects 


| know | 
ean I can 
iesin apply I think 
the | believe | what | maths or 
material can be learn in science 
Coefficient Name we Sig. euceeceall Sig. ete Sig. |. velee Sig. 
' in maths or or important 
cover in ; . 
mathe science science to my 
pe (Coef.) class to future 
Scionee real life (Coef.) 
(Coef.) (Coef.) 
After treatment -0.06 si -0.05 . 0.17 said 0.02 
(0.02) (0.02) (0.02) (0.02) 
Student fixed 
effects yes yes yes yes 
R2 0.75 0.80 0.79 0.83 
Mean in pre- 
treatment 4.66 4.39 3.90 4.38 
N 2,029 2,007 1,993 2,016 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
high-achieving girls with complete data (those who responded to the survey questions before exercise 1 
and then again after exercise 3). Standard errors are reported in parenthesis. * p $ 0.05 ** p< 0.01 ***ps 
0.001 


Table 7 shows that after treatment there was a significant decrease (5 ppt) in reported 
discussion with parents/guardians about the importance of maths or science in the last 
month: the proportion of girls reporting having talked to a parent/guardian on this subject 
at least once in the last month decreased by about 7% (from 75% to 70%) 


There was no significant change in reported discussion with parents/guardians about A 
Level choices in the last month. 
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Table 7 Overall treatment effect on self-reported discussions with parents/guardians 


| have had at least | have talked to a 
one conversation parent or guardian at 
mee with a parent or ; least once about the . 
SME CT guardian about my A alg, importance of maths alg, 
level choices in the or science in the last 
last month (Coef.) month (Coef.) 
After treatment -0.01 -0.05 ae 
(0.01) (0.01) 
Student fixed effects yes yes 
R2 0.77 0.75 
Mean in pre-treatment 0.56 0.39 
N 2,014 2,018 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
high-achieving girls with complete data (those who responded to the survey questions before exercise 1 
and then again after exercise 3). Standard errors are reported in parenthesis. * p $ 0.05 **p < 0.01 ***ps 
0.001. Annex Tables 3 and 4 show the same analyses for ‘very high-achieving’ girls only. The only 
significant change after treatment was an increase in agreement with the statement ‘I can apply what | 
learn in maths or science class to real life’ (a 0.19 point increase in the 1 to 6 agreement scale, p < 0.07). 


6.4.2. Comparing different treatments 


Table 8 shows that there was a more positive change in high-achieving girls’ reported 
discussions with parents/guardians about A Level choices for schools that received the 
parent-focused intervention as well (as compared to the schools that received the 
student-focused intervention alone). 


There were no other significant differences between the groups of schools in the changes 
after treatment in these secondary outcomes, as can be seen in table 9. 


Annex Tables 5 and 6 show the same analyses for ‘very high-achieving’ girls only. For 
this smaller subset, no significant differences were found between the groups of schools 
in the changes after treatment in these secondary outcome measures. 
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Table 8 Relative impact of different treatments on self-reported discussions with 


parents/guardians 


Coefficient Name 


| have had at least one 
conversation with a 
parent or guardian 
about my A level 
choices in the last 


Sig. 


| have talked to a parent 
or guardian at least 
once about the 
importance of maths or 
science in the last 


Sig. 


month month 

(Coef.) (Coef.) 
After treatment -0.05 = -0.05 ics 

(0.02) (0.02) 
Parent-focused 0.47 -0.50 
intervention 

(0.34) (0.34) 
After treatment * 
Parent-focused 0.07 0.00 
intervention 

(0.02) (0.02) 
Student fixed 
effects yes yes 
R2 0.77 0.75 
Mean in pre- 
treatment 0.56 0.39 
N 2,014 2,018 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
high-achieving girls with complete data (those who responded to the survey questions before exercise 1 
and then again after exercise 3). Standard errors are reported in parenthesis. * p $ 0.05 ** p < 0.01 ***ps 


0.001 
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Table 9 Relative impact of different treatments on expectations of success and the 
perceived utility of STEM subjects 


| know | lean anol | think 
can learn | believe | PPly maths or 
what | learn % 
the can be : science 
ae : in maths or . 
Coefficient material we Si successful Si Selance Si will be Si 
Name cover in 9: in maths g- Glass €6 g- important 9: 
maths or or science : to my 
: real life 
science (Coef.) (Coef.) future 
(Coef.) ; (Coef.) 
After treatment -0.07 7 -0.04 0.20 7% 0.01 
(0.03) (0.03) (0.04) (0.04) 
Parent-focused 0.49 0.51 0.02 0.01 
intervention 
(0.69) (0.71) (0.78) (0.76) 
After treatment 
poten 0.01 0.01 -0.05 0.02 
focused 
intervention 
(0.04) (0.04) (0.05) (0.05) 
Student fixed 
effects yes yes yes yes 
R2 0.75 0.80 0.79 0.83 
Mean in pre- 
treatment 4.66 4.39 3.90 4.38 
N 2,029 2,007 1,993 2,016 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
high-achieving girls with complete data (those who responded to the survey questions before exercise 1 
and then again after exercise 3). Standard errors are reported in parenthesis. * p $ 0.05 **p < 0.01 ***ps 


0.001 


6.4.3. Differential effect on girls when compared to boys for the primary 


outcomes 


Table 10 shows that the change in stated intentions to study STEM subjects after 
treatment was not significantly different in high-achieving girls and high-achieving boys. 


The positive ‘Boys’ coefficients seen for physics and computing highlight that boys were 
twice as likely to express an intention to take both physics and computer science prior to 
any intervention, once student fixed effects were controlled for. 


These results should be treated with caution, as it is unclear whether schools have 


implemented these interventions in the same way for both boys and girls. A school’s 
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decision as to which students could engage with the interventions may introduce a 
selection bias into these results. 


Table 10 Relative treatment effect for boys and girls on subject intentions 


rae : Computer 1+ STEM 2+ STEM 
Renan: eon Sig. Fak Sig. Science Sig. subject Sig | subject Sig 
; : (Coef.) (Coef.) (Coef.) 
After 
treatment -0.02 0.00 -0.01 0.01 0.01 
(0.01) (0.01) (0.01) (0.01) (0.01) 
Boys 0.01 1.00 — 1.01 ee 0.00 0.01 
(0.28) (0.23) (0.18) (0.23) (0.26) 
After 
treatment - 0.02 0.00 0.02 -0.01 0.01 
Boys 
(0.02) (0.01) (0.01) (0.01) (0.01) 
Student fixed 
effects yes yes yes yes yes 
a 0.85 0.86 0.87 0.86 0.87 
Mean in pre- 
treatment 0.47 0.24 0.15 0.76 0.52 
N=2,640 


Data collected through a bespoke online platform. Sample restricted to the 2,640 high-achieving students 
with complete data (those who provided this information both before exercise 1 and then again after 
exercise 3). Standard errors are reported in parenthesis. * p < 0.05 ** p < 0.01 *** p< 0.001 


6.4.4. Comparison of the treated year group with the previous, untreated, 
year group 


This analysis compares, within the same schools, the formal STEM A Level subject 
preferences amongst students in the ‘treated’ year group with the STEM A Level uptake 
amongst students in the previous year group (none of whom received the interventions). 
Table 11 shows that there were no significant differences in the final STEM A Level 
subject choices (as submitted by schools) between the treated cohort and the preceding, 
untreated, cohort. 
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Only schools with sixth forms that supplied a complete set of data for both cohorts were 
included in this analysis; this reduced the sample to only 30 schools. 


The lack of statistically significant findings may be due in part to the low sample size. 
There are also likely to be differences between the cohorts, as listed in the assumptions 
section, which are not controlled for in the analysis; these results should therefore be 


treated with some caution. 


Table 11 Cross-cohort analysis, overall treatment effect 


% girls % girls 
% girls % girls % girls choosing choosing 
Coefficient choosing : choosing | ., choosing : aes E tworeh : 
Name Maths Sig. Physics Sig. Computing Sig. | more Sig. | more Sig. 
(Coef.) (Coef.) (Coef.) DUELS EL 
: : : Level Level 
(Coef.) (Coef.) 
Treated 
cohort 0.02 0.01 0.01 -0.04 0.00 
(0.03) (0.01) (0.01) (0.04) (0.02) 
Cohort Year 
Group 11 
(base line 
year 10) -0.09 0.02 0.02 0.11 0.08 
(0.12) (0.04) (0.02) (0.14) (0.07) 
Percentage 
of girls in 
cohort -0.02 0.08 -0.02 -0.20 0.05 
(0.44) (0.14) 0.08 0.51 0.27 
School Type 
fixed effects yes yes es es es 
School FE Yes Yes Yes Yes Yes 
R2 0.57 0.70 0.74 0.76 0.80 
Mean in pre- 
treatment 0.23 0.06 0.02 0.38 0.21 
N 30 30 30 30 30 


Data submitted by schools. Sample restricted to the 30 schools with sixth forms that provided data for both 
the treated cohort and the preceding cohort. Standard errors are reported in parenthesis. * p< 0.05 **ps 
0.01 *** p < 0.001 


Table 12 shows that no significant differences in the cross-cohort analysis were found for 
schools that received the parent-focused intervention in addition to the student-focused 


one. 


47 


Table 12 Cross-cohort analysis, comparing different treatments 


C1) 1 0 1 
% girls % girls % girls sl ae cele 
Coefficient choosing Si choosing Si choosing Si ae sy ren tSlallciwaror ai ‘ Si 
Name Maths g Physics g Computing De one g ene g 
(Coef.) (Coef.) (Coef.) EES Ue 
: i : Level (Coef.) Level (Coef.) 

Treated cohort -0.07 -0.02 -0.01 -0.01 0.00 

(0.04) (0.01) (0.01) (0.05) (0.03) 
Parent-focused 
intervention 0.02 -0.01 0.01 0.41 a 0.20 * 

(0.11) (0.04) (0.02) (0.14) (0.08) 
Treated cohort * 
parent-focused 
intervention 0.12 0.02 0.01 0.09 0.00 

(0.06) (0.02) (0.01) (0.07) (0.04) 
Cohort Year 
Group (base line 
year 10) -0.17 0.02 0.00 -0.34 -0.12 

(0.16) (0.06) (0.03) (0.19) (0.11) 
Percentage of 
girls in cohort 0.02 0.09 -0.01 -0.16 0.06 

(0.42) (0.14) (0.08) (0.5) (0.28) 
School Type 
fixed effects yes yes yes yes yes 
R2 0.63 0.71 0.75 0.78 0.80 
Mean in pre- 
treatment 0.23 0.06 0.02 0.38 0.21 
N 30 30 30 30 30 


Data submitted by schools. Sample restricted to the 30 schools with sixth forms that provided data for both 
the treated cohort and the preceding cohort. Standard errors are reported in parenthesis. *p < 0.05 **ps 
0.01 *** p < 0.001 


6.4.5. Correlation between the intention to study STEM subjects and 


actual A Level choices 


Students’ formally-submitted A Level choices, provided by schools, were compared to 
students’ stated intentions when surveyed at the end of the trial. 


Table 13 shows that intention and actual choice are strongly correlated. A student stating 
that they intend to choose a STEM subject is almost five times more likely to enrol in this 
subject, compared to a student who did not intend to do so. Of the high-achieving girls 
who stated an intention to take at least one STEM A level 73% went on to take one. 


Overall, not everyone who expressed intention to study a STEM subject selected a 
STEM subject as a final choice. This was also the case for non-STEM subjects. 
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Table 13 Correlation between stated intention to study STEM subjects and actual A Level subject choices 


Coefficient Name Maths (Coef.) | Sig. | Physics (Coef.) | Sig. Sab sig. | 1* Sess Sig. | 2* ee aes Sig. 
Intention to study 0.50 aad 0.39 aad 0.43 a 0.53 vid 0.52 ida 
subject at A-level 
(0.02) (0.02) (0.02) (0.03) (0.02) 
Ever FSM -0.05 -0.01 -0.04 em -0.07 -0.03 
(0.04) (0.03) (0.01) (0.04) (0.03) 
Cohort Year Group 11 0.01 0.01 0.00 0.11 ae 0.05 
(base line year 10) 
(0.03) (0.02) (0.01) (0.04) (0.03) 
Predicted 
Achievement 
High achieving 0.05 -0.04 -0.01 0.07 -0.03 
0.05 0.03 0.02 0.05 0.04 
Very high achieving 0.13 * -0.01 0.00 0.15 i 0.10 * 
0.05 0.04 0.02 0.06 0.05 
Ethnicity fp 
Irish -0.20 0.04 0.00 -0.31 * -0.10 
(0.12) (0.08) (0.05) (0.13) (0.12) 
pe dele a 0.06 0.07 . 0.04 ’ 0.07 0.07 
ackground 
(0.05) (0.03) (0.02) (0.05) (0.05) 
Sis lea 0.07 0.16 * -0.05 0.04 0.08 
aribbean 
(0.1) (0.07) (0.04) (0.1) (0.09) 
White and Black African -0.16 -0.05 -0.03 -0.02 -0.22 . 
(0.11) (0.07) (0.04) (0.12) (0.1) 
White and Asian 0.07 -0.01 0.03 0.09 0.06 
(0.09) (0.06) (0.03) (0.09) (0.08) 
Any other Mixed / 
Multiple ethnic 0.08 0.03 -0.02 0.11 0.13 
background 
(0.09) (0.06) (0.04) (0.1) (0.09) 
Indian 0.11 e 0.05 0.00 0.24 ae 0.20 as 
(0.04) (0.03) (0.02) (0.05) (0.04) 
Pakistani 0.03 0.01 -0.03 0.21 an 0.25 bila 
(0.06) (0.04) (0.02) (0.06) (0.06) 
Bangladeshi 0.27 =. 0.17 = 0.06 . 0.24 a 0.34 — 
(0.08) (0.05) (0.03) (0.09) (0.08) 
Chinese 0.10 -0.02 0.03 0.28 7 0.17 
(0.1) (0.07) (0.04) (0.11) (0.1) 
Al Rall nial 0.15 : 0.11 : 0.09 = 0.16 : 0.26 - 
background 
(0.06) (0.04) (0.02) (0.07) (0.06) 
African 0.20 oe -0.02 0.00 0.14 * 0.21 es 
(0.05) (0.04) (0.02) (0.06) (0.05) 
Caribbean 0.02 -0.10 0.00 -0.04 -0.03 
(0.17) (0.12) (0.07) (0.18) (0.16) 
Any other Black / 
African / Caribbean 0.21 0.17 -0.05 0.10 -0.02 
background 
(0.14) (0.09) (0.05) (0.15) (0.13) 
Any other ethnic group -0.17 0.03 -0.03 -0.12 -0.07 
(0.1) (0.07) (0.04) (0.11) (0.09) 
R2 0.34 0.24 0.39 0.33 0.42 
N 1,318 1,318 1,318 1,318 1,318 


Data collected from a bespoke online platform and submitted by schools. Sample restricted to the 639 high-achieving girls for 


whom there was both survey data and actual subject choices available. Standard errors are reported in parenthesis. * p < 0.05 ** p 


< 0.01 *** p< 0.001 


7. Discussion 


7.1. Intervention impact 


The aim of the interventions in this study was to increase high-achieving girls’ uptake of 
STEM A Levels. After the interventions, high-achieving girls were significantly more likely 
to state the intention to study two or more STEM A Levels (a 2 percentage point 
increase, p$0.05). This positive change was significantly greater for high-achieving girls 
in schools that received the parent-focused intervention in addition to the student-focused 
intervention (by 4 percentage points, ps0.05). Uptake of two or more STEM A Levels was 
an outcome of particular interest for this research because this is a pre-requisite for the 
majority of university STEM pathways in which girls are currently under-represented. 


The principle mechanism by which the interventions sought to increase girls’ uptake of 
STEM A Levels was by improving perceptions of STEM’s utility — this was the target of 
the parent-focused intervention, as well as two of the three student-focused intervention 
exercises. After the interventions, high-achieving girls expressed significantly greater 
agreement with the statement “I can apply what | learn in maths or science class to real 
life” — a measure of the perceived utility of STEM (a 0.17 point increase in the 1 to 6 
agreement scale, ps0.001). 


One of the two student-focused intervention exercises sought to increase students’ 
expectations of success in STEM. However, after the interventions, high-achieving girls 
expressed significantly less agreement with the statements “I know | can learn the 
material we cover in maths or science” (a 0.06 point decrease in the 1 to 6 agreement 
scale, p$0.01) and “I believe | can be successful in maths or science” (a 0.05 point 
decrease in the 1 to 6 agreement scale, ps0.05). Given the nature of the research 
design, the reason for these small changes is not clear. While it is conceivable that the 
intervention had an unintended backfire effect on expectations of success in STEM, an 
alternative explanation is that students tend to become less confident in their ability in 
STEM over time, as they get closer to their GCSE exams. Indeed, it is even possible that 
the intervention counteracted a natural decline that would, otherwise, have been larger. 


After the interventions, as compared to before the interventions, high-achieving girls were 
significantly less likely to report having discussed the importance of maths or science 
with their parents in the last month (a 5 percentage point decrease, ps0.001). One 
possible explanation for this finding is that the month prior to the interventions contained 
a school holiday, affording more opportunities for conversations with parents, whereas 
the month preceding the end of the interventions did not. However, high-achieving girls in 
schools that received the parent-focused intervention in addition to the student-focused 
intervention showed a relative increase in self-reported discussions with their parents 
about A Level choices (by 7 percentage points, p<0.01). 


It is important to note that no causal inferences can be made between the activities 
run for students and parents and the changes recorded in primary and secondary 
outcome measures. There are many external factors that are likely to influence 
students’ perceptions of STEM subjects and indeed their final A Level choices and 
without a viable control group for comparison, it is difficult to separate the impacts 
of these external influences from the trial itself. 


There was a strong relationship between the intentions of girls to study STEM subjects 
with their final A Level choices. While this is reassuring, not all stated intentions to study 
STEM A Levels translated into formally-submitted A Level preferences (as was also the 
case for non-STEM A Levels). This may be explained by a natural intention-action gap, 
meaning that there is a difference between what people say they plan to do and what 
they actually do. It is also possible that the design of the surveys had an impact on this 
outcome. In the surveys, students could select more subjects than the standard number 
of A Levels for which students generally enrol. As a result, some students may have 
included more subjects in their stated intentions than it was actually feasible for them to 
choose. 


We need to be similarly cautious when interpreting the comparisons made between the 
two treatment groups. The analysis was severely underpowered, and it cannot be 
excluded that there were no other confounding factors affecting the observed changes in 
the survey responses of each group. Nevertheless, students benefiting from parental- 
focused activities did show an increased likelihood of discussing A Level choices with 
their parent or guardian after the trial, and this is something which clearly warrants further 
exploration. 


This research sought to have an immediate impact on girls’ A Level choices. However, 
the longer-term effects of societal norms or general expectations about the challenges of 
the subject matter itself may require that interventions are targeted much earlier, 
particularly for girls. 


7.2. Limitations 


e School dropout 


Despite having initially agreed to take part in the intervention, some schools dropped out 
and decided not to implement the intervention (as is typical in trials). This had two potential 
consequences: 1) schools may have disproportionately dropped out from one of the two 
treatments. If attrition was non-random across the two groups, the comparison of student 
only and student+parent intervention may suffer from selection bias. For analyses that do 
not test the differential impact of the two treatments, this does not cause a problem per se; 
2) schools who took part in the intervention may have been a selected sample with 
particular characteristics for which the treatments may be more or less beneficial with 
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respect to the general population of schools. This may affect the generalisability of the 
results. 


A simple comparison of the characteristics of students in the sample (as reported in the 
Descriptive Statistics section, Figure 1) to national figures shows that the treated sample 
is not nationally representative. For instance, the FSM rate in secondary schools in 
England"? is 14.1%, while the rate in the sample is 12%. Similarly, schools in the North 
West are overrepresented in the sample at 21%, while 14% of state-funded secondary 
schools are in the North West as of 20197°. 


The schools were free to administer the interventions to whomever they chose within the 
target year group. They were asked to administer them to every student or to high- 
attaining students only, as they saw fit. This has three potential consequences. 


First, if schools choose to deliver the intervention only to students who may benefit the 
most from the intervention, the effect of treatment may have been overestimated. 


Second, student selection implies that the cross-cohort analysis estimates are to be 
interpreted as intention to treat, rather than average treatment effect. 


Third, it is not certain that the girls and boys who took part in the activities were matched, 
or representative of the schools, in terms of achievement level. This complicates the 
interpretation of gender difference analysis, as attainment is known to be a moderator of 
the impact of interventions of this kind, and to operate differently for girls and boys". 


e Student-level attrition 


Dropout of students from later stages of the intervention raises concerns for the 
generalisability of the evaluation. It is possible that students who dropped out were the 
ones for which the treatment would have had the smallest/greatest effect. 


The reduction in student numbers also impacts the power of the statistical models 
produced, reducing the minimum change in survey responses it was possible to detect. 


e The lack of syncing between the parent and student interventions 


The timing of the administration of the student intervention activities was at the schools’ 
discretion. For some schools, the initial communication of the parent intervention may 
have been delivered prior to the first student intervention activity (and, therefore, before 
the ‘pre’ measures). Furthermore, some schools may have delivered the third student 
intervention activity (which included the post-survey with the final outcomes measures) 


12 DfE, Percentage of pupils eligible for and claiming free school meals (FSM) in England, 2011-2019 (link) 
13 DfE, Schools, Pupils and their Characteristics: January 2019 (link) 

14 Rozek CS, Hyde JS, Svoboda RC, Hulleman CS, Harackiewicz JM (2015) Gender differences in the 
effects of a utility-value intervention to help parents motivate adolescents in mathematics and science. J 
Educ Psychol 107(1):195-206. 
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before all parental correspondence has been completed. This implies that the true effect 
of the parent-focused intervention may be underestimated. 


e Technical difficulties 


The intervention implementation was not entirely smooth during the early stages. For 
example, there were some complications with the parent website, and the video 
testimonials from students (about the ways in which STEM has been useful in their lives) 
may have been inaccessible for at least part of the intervention period. This implies that 
the treatment effects are considered as intention to treat. 


e Prompts within the student-focused intervention to talk with parents 


Prompts to speak with a parent/carer about post-16 choices and the value of STEM were 
written into the student-focused interventions when the research design was an 
individual-level RCT with the following treatment conditions: control; parent-focused 
intervention only; parent-focused intervention + student-focused intervention. However, 
when the research design was changed (to be a pre-post study design with all schools 
receiving the student-focused intervention and half of schools additionally receiving the 
parent-focused intervention) these prompts were not removed from the student-focused 
intervention. These prompts may well have influenced parent-child conversations about 
the value of STEM in the student-focused intervention only condition, making it relatively 
harder to see an added effect of the parent-focused intervention (when the comparison 
was already underpowered due to the change in design). 
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8. Recommendations 


8.1. Future research 


Without a robust control group, it is difficult to be certain of what would have happened in 
the absence of the interventions. Therefore, the effectiveness of the interventions 
remains uncertain. Future research incorporating a valid control group, ideally from a 
randomised control trial, would be very valuable. Given the department's preference for 
randomising at school level to ensure students within a school are not unfairly 
disadvantaged by different provision, a larger sample of schools would be required. This 
future research could have the added benefit of using actual A Level subjects taken as 
the outcome measure, which was outside the scope of the timeline of the current project. 


The effects of the novel coronavirus (COVID-19) impacts the recommendations for 
further research in the short-term. In light of the disruption of teaching, examination and 
assessment in 2020" and the Ofqual consultation to consider a range of possible 
measures to free up teaching time’®, education providers will need to prioritise covering 
core curriculum content and supporting young people to progress to further study. 
Therefore, further research employing the student-focused intervention exercises in 
school hours is unlikely to be feasible in the coming months. 


This may however open up opportunities to carry out research on the impact that COVID- 
19-related changes are having on STEM uptake (compared to other subjects); provided 
the regional effects of any changes are uniform across the entire student cohort in 
England, sampling conditions would be ideal. Additionally, the prevalence of online 
learning and accessing resources remotely throughout the pandemic may provide 
additional benefits to delivering an at-home online intervention, as young people would 
be more comfortable and familiar with learning in this way. It may also be a way to 
reduce the administrative burden on participating schools. Further research using the 
parent-focused intervention could also be feasible, as this places no burden on teaching 
time. 


The present research sought to have an immediate impact on girls’ A Level choices. 
However, it would be remiss to omit that there is a wealth of evidence (McGuire et al., 
2019; Sullivan, 2019; Van Den Hurk et al., 2019) for starting STEM interventions much 
earlier. It may be too late to generate a long-term impact on female students’ perception 
of STEM just before they are about to choose their A Level subjects. The perceived value 
of STEM and students’ expectations of personal success in these subjects must be 
consistently built upon, and success and achievement in STEM subjects across both 
genders needs to be normalised from an early age. The recommendation would be to 


18 DfE Guidance Coronavirus (COVID-19): cancellation of GCSEs, AS and A levels in 2020 (link) 
16 Ofqual launches consultation on 2021 exams and assessments (link) 
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start interventions earlier, in addition to a timely nudge just before students make their 
further education decisions. 


8.1.1. The influence of other research on future plans 


e ASPIRES 


While this research was ongoing, the research team for ASPIRES released the report for 
the second wave of the research — ASPIRES 2. The findings of the ASPIRES 2 research 
was used to understand some of the outcomes from this research. When planning any 
future work, it would be useful to draw upon some of theories posited, specifically those 
examining influences for ‘careers education’ and ‘masculine science’ and ‘clever 
science’. It would also be useful to examine capital-related inequalities which the 
research did not explicitly address. 


At the time of publishing, the next phase of the research — ASPIRES 3 — has been 
initiated and seeks to build on previous research. The ASPIRES team will continue to 
track the same cohort of young people in order to ‘understand the changing influences of 
the family, school, careers education and social identities and inequalities on young 
people's science and career aspirations’. 


e Science Education Tracker 


Similarly, the Wellcome Trust published its findings on young people’s views on science 
education and careers. This was based on a survey of more than 6,400 students in years 
7-13 (aged 11-18) in schools and colleges across England. Among other key findings, 
the survey found that many young people do not see science as relevant to their 
everyday lives or future plans. Female students in years 10-13 were found to be less 
likely than male students to rate themselves as good at maths (63% males, 51% 
females) or physics (46% males, 28% females). 


e PISA 2018 


PISA 2018 found that while English students outperform counterparts in OECD countries 
in maths, the gender gap is larger, and the pattern is the opposite to that seen in national 
tests such as GCSEs. Finland was noted as a notable exception to this trend, with girls 
outperforming boys. Positive findings indicate that in science, both girls and boys 
outperformed counterparts in the OECD, however, once again girls in Finland 
outperformed boys by a large margin. Perhaps it would be useful to understand practices 
in this country and see whether they can be applied within this context. Interestingly, 
there will be a focus on science in PISA 2024 with an aim to better understand levels of 
scientific literacy and how science is applied in a real-life context. 
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e Public Attitudes to Science (PAS) 


The recently published PAS report also gives us a good measure of the UK population’s 
perception of and engagement with science, which is useful when planning future 
research, particularly around the utility value of STEM. Science was seen as a ‘broad and 
ambiguous topic synonymous with technological advancement’. There was a broad 
agreement about the importance of science in our lives, particularly in terms of 
healthcare. This finding may also be an interesting area to explore in terms of whether 
there is any correlation between perception of healthcare for girls and their higher levels 
of uptake of biology and chemistry than boys. There was widespread agreement that 
science plays a fundamental role in society and key to improving it which is also 
consistent with the ASPIRES 2 finding that persistent, low science aspirations are not 
due to a lack of interest in science (Archer, 2020). 


8.2. Recommended adaptations for the present interventions 


8.2.1. Targeting 


Where possible, interventions of this kind should be delivered to a targeted group — those 
who are expected to be eligible for STEM A Levels. First, this would help to personalise 
the intervention; personalisation is known to capture attention and promote behaviour 
change. '’ Second, it would allow the interventions to allude to the student’s success in 
STEM subjects to date, helping to increase expectations of success if they were to 
continue with these subjects at A Level. Third, it would mean that these interventions, 
including the correspondence to parents, could represent specific encouragement for the 
selected students, which is associated with continued participation in STEM subjects, '® 
and a gender disparity in encouragement is thought to be at least partly responsible for 
the gender gap in STEM participation. 1 Finally, it would help ensure that the 
interventions are relevant and suitable for all students receiving them. However, the way 
that such interventions are presented and framed is important: if girls feel singled out on 
the basis of their gender or receive the message that girls are currently underrepresented 
in STEM fields, this could potentially backfire. 


17 The Behavioural Insights Team (2014). EAST: Four Simple Ways to Apply Behavioural Insights 

18 Archer, L., & Tomei, A. (2013). What influences participation in science and mathematics. A briefing 
paper from the Economic and Social Research Council (ESRC) Targeted Initiative on Science and 
Mathematics Education (TISME)(ESRC, Swindon). 

18 Newall, C., Gonsalkorale, K., Walker, E., Forbes, G. A., Highfield, K., & Sweller, N. (2018). Science 
education: Adult biases because of the child’s gender and gender stereotypicality. Contemporary 
Educational Psychology, 55, 30-41. 

Accenture (2017) Girls in STEM 

Ipsos MORI (2019) Omnibus survey of pupils and their parents or carers: wave 5 
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If interventions are not targeted, the recommendation would be to expand the 
interventions to pertain to a variety of post-16 STEM routes, rather than STEM A Levels 
exclusively, and that the student interventions are video based, rather than text based, to 
increase their accessibility. 


It should be noted that interventions delivered at an earlier stage of development would 
require less targeting, as there would be more time for attainment to improve. 


8.2.2. Timing 


In this project, the interventions were implemented during the spring term. This meant 
that it was necessary to work with year 10 students in schools in which year 11 students 
had already been asked to indicate their A Level subject preferences. As mentioned 
above, future research could seek to intervene earlier in students’ development, in 
conjunction with timely nudges just before students make their decisions. For STEMA 
Level decisions, such timely nudges would be best delivered in autumn term in year 11. 
This would have the added benefits of giving schools more time to deliver the student 
interventions and minimizing burden on schools in the run up to exam time. 


8.2.3. Additional options 


It may be helpful to broaden the communications for the parent intervention to include 
text messages and/or physical post, as these are likely to be more salient options for 
some parents. 


The parent website in this project focused heavily on boosting parents’ perceptions of the 
utility of STEM for their child. However, it could, in future iterations, be extended in a 
variety of ways. For example, messages to increase the sense that their child belongs in 
STEM, and to boost confidence, could be built in. 
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Annex A 


School-level randomisation balance checks 


Table 14 Characteristics of students sampled 


Variable Arm 1 Arm 2 P-value of the 
difference 
Share of FSM students 0.224 0.184 0.310 
Share of students whose 0.690 0.636 0.477 
ethnicity is white 
Average expected score in 5.538 5.741 0.382 
GCSEs Maths 
Average expected score in 5.540 5.758 0.339 
GCSEs Maths, girls only 
Share of girls with imputed 0.030 0.062 0.468 
expected GCSE 
Share of girls in the year- 0.599 0.621 0.680 
group 
Share of girls in the school 0.559 0.548 0.806 
Year group size 169.914 168.242 0.921 
Finishing age 17.429 1515 0.691 
School is LA maintained 0.257 0.333 0.498 
School is single gender 0.143 0.152 0.921 
School is non-religious 0.800 0.758 0.679 
School is in London 0.143 0.212 0.461 


20 The p-value of the differences between the two arms are greater than 0.05 and are therefore not 
statistically significant at the 95% level. 
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Annex B 


Survey measures 


1. If you were to take AS and A Levels, and if you had to make your subject choices right 
now, how sure would you feel of your choices? 

(a) Sure 

(b) Somewhat sure 

(c) Not sure at all 


2. If you answered (a) or (b), which subjects would you choose (please select up to five)? 
[NB the subjects were presented in a random order] 


English literature 
Spanish 

Biology 

Chemistry 

Geography 

History 

Physics 
Computing/computer science 
Further maths 
German 

Art and design 
Business studies 
French 

Sociology 

Psychology 

Maths 

Other (please specify): 
Other (please specify): 
Other (please specify): 
Other (please specify): 
Other (please specify): 


Please read the following statements and indicate the extent to which you agree or 
disagree with each. Remember, please respond openly and honestly; your answers won't 
be shared with your school. 


1. | know | can learn the material we cover in maths or science 
Strongly disagee 

Disagree 

Slightly disagree 

Slightly agree 

Agree 

Strongly agree 
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2. | believe | can be successful in maths or science 
Strongly disagee 
Disagree 
Slightly disagree 
Slightly agree 
Agree 


Strongly agree 


3. | can apply what | learn in maths or science class to real life 
Strongly disagee 


Disagree 
Slightly disagree 
Slightly agree 
Agree 
Strongly agree 


4. | think maths or science will be important to my future 
Strongly disagree 


Disagree 
Slightly disagree 
Slightly agree 
Agree 

Strongly agree 


5. | have had at least one conversation with a parent or guardian about my A Level 
choices in the last month 
Yes 


Not sure 
No 


6. | have talked to a parent or guardian at least once about the importance of maths or 
science in the last month 
Yes 


Not sure 
No 
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Annex C 


Interventions 


Student-focused intervention 
Overarching intro Text 


Over the coming weeks you will have some time in class to work through the following 
three exercises. These have been designed to help students like yourself reflect on your 
school life and explore links to your wider goals and future choices. Each exercise will 
take around 30 minutes. 


Many students in schools across the country are also completing these tasks. 


It is important that you answer honestly and try your best to complete each exercise in 
full. Your answers will not be shared with your teachers or school and there are no right 
or wrong answers. You also do not need to worry about spelling, punctuation or grammar 
for this exercise; just get your thoughts down. 


If you have any questions about the exercises either in or outside class please ask a 
teacher. 


Exercise 1 Expectancy 
Designed to counteract girls’ tendency to believe that they are not ‘smart’ enough to do 


these subjects (despite performing equally well at them) 
Section 1 


What research and other students say about how to do well in school. 


In this exercise, you will get the opportunity to support fellow students to achieve their 
potential as they approach their GCSEs and decide what to do next. It will start with 
information from research, followed by quotations from other students, and then you will 
be invited to contribute your own message for future students. 


You will see that many of the examples relate to Maths, Physics, Chemistry, Biology and 
Computer Science (known as STEM subjects). This is because: 


1) These subjects open doors to a wide range of education and career 
opportunities; 


2) People with skills in STEM play an important role in making a positive 
difference to people’s lives and the world we live in, as the world relies more 
and more on technology; 


3) Skills and information learnt in STEM subjects are very relevant to everyday 


life; they can be used to help improve your health and well-being, your financial 
situation and your understanding of the world around you. 
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Three lessons for achieving success 


Some people underestimate their ability in maths, science and technology. They think 
they can’t do it when they really can, and this holds them back from fulfilling their 
potential. By remembering three lessons, people can avoid this happening to them. 


1. Effort is the most important factor for success and everybody can improve. 


Some people believe you are either naturally good at maths or science or you're not, so 
effort doesn’t make a difference. This is not the case; there is a lot of evidence to 
show how much of a difference effort can make. 


Research shows that putting in effort to learn something builds and strengthens 
connections in the brain, making us better at the task the next time we try. 


This combines with evidence to show that when people believe they can get better at a 
subject through effort, they try harder, challenge themselves, carry on when things are 
difficult, learn from feedback, ask for help and...ultimately...they do become much better 
at the subject. 


Every time | work hard at maths, | 
strengthen connections in my I'm no good at maths, so there is 
no point spending time trying 

Wags 


to improve. 


xX 


brain, and learn how to perform 
better in class! 


g=> 
a 
S| 


Remember: If you put effort into science and maths, you wil/ improve. 


2. Don’t judge your performance against other people 


It is common for students at different schools to perform equally well at science and 
maths but judge their performance differently. This is because they are comparing 
themselves to their classmates, which isn’t a good idea as it can result in students 
wrongly believing they are less capable than they are (just like the person on the right in 
the image below). 
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Hmm...| got a grade 6 in 
my mock maths exam 
when most people in my 
class got a grade 7 or 
above. | can’t be very 

good at maths. 


Hooray! | got a grade 6 in 
my mock maths exam when 
most people in my class got 
a grade 5 or below. | must 
be good at maths! 


ir ol 

This matters, as beliefs influence performance. People who think they are bad ata 
subject often don’t put in as much effort and get distracted by thoughts of failure. This 
tends to make them perform worse than they otherwise would. However, when people 


believe that they can do well, they put in effort and find it easier to concentrate, which 
helps them to learn more and do better. 


Remember: Thinking you can do something can actually make this a reality. 


3. Track progress in that subject over time 


Another common mistake people make when considering their performance in maths, 
science or technology is to compare how well they are doing in these subjects to how 
well they are doing in other subjects. 


| got a grade 7 in my mock 
maths GCSE, but | got a 
grade 8 in English, so | 
can’t be very good at 
maths. 


| got a grade 7 in my mock 
maths GCSE. This is better 
than the grade 6 | got in 
English, so | must be good 
at maths. 


re el 
Some people mistakenly think that students are unlikely to be good at verbal subjects, 
like English, as well as maths, science and technology. This is not true! 


Doing well at English or humanities does not mean that maths and science are not ‘for 
you’, nor the other way around. 


Many schools require a grade 6 at GCSE for students to take a STEM subject at A Level. 
If someone is on track for this, they must, therefore, be doing unusually well in that 
subjec 


Remember: Don’t make comparisons to other people or to yourself in other 
subjects -- keep the focus on your own progress in one subject over time. 


Section 2. 


Share what you know with other students. 


As an example, here is a message from a current A Level student who is studying 
Biology, Chemistry, Government & Politics, and Economics: 


“When you're studying Chemistry, for example, don’t think about the person sitting next 
to you and what they’re getting or the person behind you and what they’re getting. Think 
about what you’re getting out of the subject. 


Immerse yourself in what you do and keep asking why - keep probing your brain to learn 
further and do more research and gain the knowledge. 


Most importantly, never give up. For me personally, taking STEM [subjects related to 
Science, Technology, Engineering and Maths] isn’t exactly easy, and there are going to 
be times when you feel like you can’t do it, but the only way to get success is to get up 
after failure. It doesn’t matter if you found a topic really hard. All you need to do is go 
back over it and think, “What could | do to learn better?” Or go and ask someone for help 
- someone who understood it better. And in that way, when you keep improving, you will 
be successful. Success isn't the first time you try and you succeed. Failure is only failure 
when you don't get back up again.” 


Messages from current students can help future students do better. Please write a 
message to help younger students believe in their ability to do well in science, maths 
and/or technology. 


Try your best; if you agree to it, your response could be added to a bank of messages, 
like the one you just read - from students to students - and used to inspire other people. 


Please write your own message, for younger students, in the box below. 

Can you include the three main points from the previous section? You can press ‘Back’ to 
remind yourself of these three main points at any time. Can you draw on examples from 
your own experience (e.g. when effort has paid off for you)? 


Please spend about 5-10 minutes on your answer. Don’t worry about spelling, grammar 
or punctuation in this exercise - any mistakes will be automatically corrected afterwards; 
just get your thoughts down. 


[untick if don’t want your answer to be shared with future students] 
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Exercise 2 Utility 
Designed to counteract girls’ tendency to believe that STEM is not ‘for them’ and is not 


relevant for their future 


Section 1: Recap 


The previous exercise in this series was about people’s beliefs in their own abilities in 
STEM subjects. STEM subjects are those related to science, technology, engineering 
and maths. 


To recap, you read about evidence which shows that some students mistakenly think 
they cannot do well in these subjects and that this can hold them back. 


In the box below, please list two or three things that people should do to help 
themselves believe in their own abilities in these subjects: 


Some of the things a student can do to make sure they don’t have mistaken beliefs about 
their ability include: 


e Reminding themselves of the importance of effort; recognising that abilities in these 
subjects are not fixed and that the most important things for success are actually 
hard work, preparation and self-confidence 


e Avoiding comparing themselves to other people 

e Avoiding comparing their abilities in these subjects to their abilities in other subjects 
Remember, applying these three lessons can actually improve performance too. 
Section 2: Why it matters 


It matters if someone wrongly believes that they can’t do STEM subjects or assumes 
these subjects are not ‘for them’; it makes them more likely to miss out on the benefits of 
continuing to study them after GCSEs. In this activity, you will reflect on the relevance of 
STEM A Level subjects - maths, physics, chemistry, biology and computer science - to 
real life: how they are actually useful. 


University and jobs 


As the world becomes more and more dependent on technology, the number of careers 
that require expertise in maths, science and/or technology is rising faster than any other 
kind of career. This means that students who have A Levels in maths, science and/or 
computing are in a very good position to get highly paid jobs. 


Even if you decide that a career in these areas is not for you, maths, further maths, 
physics, chemistry and biology A Levels are among the subjects most highly regarded by 
leading universities. Students who take these subjects are attractive candidates for a 
wide range of competitive university degree places, not only those that are obviously 
related to science and maths. 
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Making a difference 


Science, maths and computer science A Levels not only open up a huge number of well 
paid careers, but also plenty of opportunity to make a real difference in the world. By 
pursuing science, maths or computer science A Levels, you could: 


Design and construct different types of building and living spaces as an architect or 
engineer 


Reduce crime by, for example, becoming a forensic psychologist or using 
technological expertise to address cyber crime 


Treat and cure patients as a doctor, nurse or dentist 
Help people with their mental health, by specialising in psychiatry 
Help generate new solutions for climate change or treatments for disease 


Educate future generations by teaching in schools or universities 


These are just a few examples, there are many more and most involve working in teams 
with talented and creative people. 


Daily life 


The things you learn in STEM A Levels are relevant for daily life. For example, they can 
improve your: 


health and wellbeing (through understanding nutrition; medical advice; how to 
avoid food poisoning, sickness, disease, infection and sun damage; how to 
improve your sleep or reach your fitness goals), 


wealth (by helping you save money, manage money and avoid scams) 
ability to interpret news media and to spot fake news. STEM A Levels also help us 


understand the world around us and give us knowledge about how things in 
everyday life work, which makes us informed citizens. 


Section 3: What other students say 


Having read some of the reasons why maths, science and technology are important in 
general, you can now read statements from former students about the ways in which they 
find these subjects useful. 


Consider if these points could apply to your own life...now, or one day in the future. 


A: “It’s really cool to be able to work in groups during lab work, and also to try to 
understand stuff in a group. It’s really good for you if you can learn how to explain a 
concept to someone else, cause that means that you’re learning and also that you 
understand it really deeply; you need to be able to do that in order to be able to explain it 
to someone else.” 
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B: “I think STEM A Levels are really relevant to the real world because they underpin 
most things in the world. Biology teaches you how everything that’s living works. 
Computing teaches you about computers and the apps that we use. | feel STEM subjects 
are really closely tied with how we live our lives.” 


C: “One of the ways I’m finding STEM A Levels useful is in how it makes you think and 
how it progresses your mind. I’m learning a lot of transferable skills for the future, such as 
statistics, problem solving and analytical skills. And that’s going to be useful for jobs 
centred in almost every industry, such as media, finance and engineering.” 


D: “In every company that you go to, there is a department for jobs that are connected to 
STEM, like finance and tech departments. But if | were to take history, there’s not a 
history department in a workplace. So it’s beneficial to do STEM subjects because there 
is a job available: 100%...you’re in need! People are looking for people who have the 
knowledge to take this generation into the next one!” 


E: “It is really inspiring and motivating for me when | can use my maths, science and tech 
skills to make a difference. I’m working on developing new materials that can be used to 
replace bone in the body. People often tell me stories from their own lives, like they’ve 
got a relative who has problems with their joints. They’re always really excited to hear 
about what | do and when it might be something which is actually used in the NHS.” 


1) Having read those quotations, take a minute to think about your own experiences. 
Have you heard similar points about these subjects made by others or had any of these 
ideas yourself before? If so, which ones and in what situation? 


2) Of the five statements, which ones can you most relate to? What is it that convinces 
you about the statements you chose? 


3) Rank how important you personally find the quotations from least to most important 
and explain your ranking in detail. 


4) Can you contribute your own statement? Write about why maths, science and/or 
technology are important to your life; think about the ways in which these subjects are, or 
could be, useful for you in terms of everyday situations, education, university and/or your 
professional life. 


Consider talking to your parent/carer about the ways that the maths, science and/or 
technology that you learn in school can be useful in life, and the value of carrying on with 
these subjects after Year 11. Talking with them may help you discover new connections 
between these subjects and the things that really matter to you. 
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Figure 5 Screenshot of an example from exercise 2 - 'What other students say’ 


EXERCISE 2 


STEP 4 OF 6 


What other students say 


Having read some of the reasons why maths, science and technology are important in 
general, you can now read statements from former students about the ways in which they 
find these subjects useful. 


Consider if these points could apply to your own life... now, or one day in the future. 


A: “It’s really cool to be able to work in groups during lab work, and also to try to 
understand stuff in a group. It’s really good for you if you can learn how to explain a 
concept to someone else, cause that means that you're learning and also that you 
understand it really deeply; you need to be able to do that in order to be able to explain it 


Exercise 3 Purpose for taking STEM 

Designed to motivate students to engage with STEM and, in particular, to counteract the 
stereotype that STEM does not involve working with others, helping people and making a 
difference to the world, which are goals especially endorsed by girls. 

Section 1. Reflection 

Think about the world, your future and what is important to you. 


Everyone thinks that the world could be improved in one way or another. Some people 
think the most important change would be less hunger, some think greater fairness, and 
others want less violence or disease. Other people want lots of other changes. Please 
take a minute to consider ways that you think the world could be improved. 

How do you think the world could be a better place? 


In the space below, write a few sentences that answer this question. This should take 
you about 5 minutes. 


Thank you for sharing your thoughts. 
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Section 2. What other students say 

Many students, girls and boys, choose to take maths, further maths, biology, chemistry, 
physics and/or computer science at A Level. These subjects are known as STEM 
subjects (which stands for Science, Technology, Engineering and Maths). 

STEM subjects can lead to a wide range of career opportunities, for example: 


e Designing and constructing different types of building and living spaces as an 
architect or engineer 


e Reducing crime by, for example, becoming a forensic psychologist or using 
technological expertise to tackle cyber crime 


e Treating and curing patients as a doctor, nurse or dentist 

e Helping people with their mental health, as a psychiatrist 

e Helping generate new solutions for climate change or treatments for disease 
e Educating future generations by teaching 


A recent survey highlights some of the less obvious reasons students choose to continue 
with STEM subjects at A Level, in addition to getting a well-paid job: 


e All students thought that they could use the knowledge and skills they learn from 
STEM A Levels to help to make a difference in the world 


e 6in 10 said that students studying STEM subjects are more likely to develop 
knowledge and skills to make a difference in the world than students studying 
humanities 


e All students believed that STEM A Levels could help them to get a job that involves 
helping people 


e All said that STEM A Levels are well-respected by universities 


e 7 in 10 said that taking STEM A Levels increases your chances of getting into a 
good university 


e All stated that STEM A Levels could help them get a job where they work in a team 


It’s often more interesting to hear about people’s experiences in their own words. Read 
on to hear what other students said about their interest in STEM. Afterwards you will be 
asked to share your own thoughts. 


“STEM subjects are those that make the greatest impact on society. It leads to 
professions such as medicine, engineering, biotechnology, economics...and these are 
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the subjects that have the greatest impact on people. Being part of this group is a great 
privilege.” 


“[STEM A Levels] provide skills that are relevant in many areas e.g. teamwork. They also 
allow progression onto courses that directly aid people e.g. research or healthcare.” 


“Tech makes our lives more efficient so that’s why | want to study towards getting a job in 
the Tech industry. It simplifies our lives.” 


“Society benefits from having more people who understand science and maths and can 
think critically. Anew invention may seem far-fetched but, in reality, people are out there 
inventing and developing techniques right now that will be changing lives in years to 
come. STEM A Levels open up the possibility of a job you might not know exists (at least 
not yet!)” 


“Current advances in technology will have a positive impact on people’s lives. We can 
improve communication, improve the lives of those with a disability and hopefully widen 
education to poorer countries.” 


Section 3. 


Time to share your thoughts 


Think about the kind of person you want to be in the future and the kind of positive impact 
you want to have on the people around you or society in general. 


Try not to concentrate on outcomes such as money, high status, or power—even though 
those things can be important. Instead, try to think about values that are important to how 
you see yourself, and how doing STEM subjects at A Level could help you to achieve 
those goals. 


To remind you, STEM stands for Science, Technology, Engineering and Maths. The 
STEM subjects available at A Level include maths (and further maths), biology, 
chemistry, physics and computer science. 


If you took STEM A Levels in the future, how could they help you become the kind 
of person you want to be or help you make the kind of impact you want on the 
people around you or society in general? 


In the space below, write a few sentences that answer this question. This should take 
you about 5-10 minutes. 


Try your best; if you agree to it, your answer could be shared with future students, like 
the quotations you have just read. 


[untick if don’t want your answer to be shared with future students] 
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Parent-focused intervention 


www.subjectchoices.co.uk 


Figure 6 Screenshot of the landing page of the website parents accessed for the parent- 
focused intervention 


< Subject Choices What youcando Theimportance of STEM | STEM A Levels Options 


Helping your child with 


id at=Melple)(est-r-lal=rle 
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Annex D 


Results of Very High-Achieving Girls 


Annex Table 1 Overall treatment effect on expectations of success and the perceived utility of 
STEM subjects for Very High Achieving Girls 


1+ 2+ 
Secilah Maths | sig | Physics | sig oe Sig. | STEM | Sig | STEM | Sig 
ame (Coef.) (Coef.) subject : subject : 

(Coef.) 

(Coef.) (Coef.) 
After 
treatment 0.00 0.01 0.01 0.02 0.00 

(0.02) (0.02) (0.01) (0.02) (0.02) 

Student 
fixed 
effects yes yes yes yes yes 
R2 0.93 0.88 0.88 0.85 0.91 
Mean in 
pre- 
treatment 0.59 0.18 0.09 0.83 0.63 
N=253 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
253 very high-achieving girls with complete data (those who responded to the survey questions before 
exercise 1 and then again after exercise 3). Standard errors are reported in parenthesis. * p< 0.05 **ps 
0.01 *** p< 0.001 
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Annex Table 2 Relative impact of different treatments on intentions to study STEM subjects for 
Very High Achieving Girls 


1+ 
Coefficient | Maths | Sig | Physics | sig | COMPUT | sig) stem |... | 2* STEM | _ 
Name (Coef.) (Coef.) Science subject Sig. | subject | Sig. 
: : : ; (Coef.) : (Coef.) 
(Coef.) 

rie 0.00 0.01 0.01 0.01 -0.02 
treatment 

(0.02) (0.02) (0.02) (0.02) (0.02) 
Parent- 
focused -0.99 ae 0.51 + 0.00 -0.02 -0.04 
intervention 

(0.18) (0.19) (0.15) (0.2) (0.21) 
After 
treatment * 
Parent- -0.01 -0.01 0.00 0.03 0.07 
focused 
intervention 

(0.03) (0.04) (0.03) (0.04) (0.04) 
Student 
fixed 
effects yes yes yes yes yes 
R2 0.93 0.88 0.88 0.86 0.91 
Mean in 
pre- 
treatment 0.59 0.18 0.09 0.83 0.63 
N=253 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
253 very high-achieving girls with complete data (those who responded to the survey questions before 
exercise 1 and then again after exercise 3). Standard errors are reported in parenthesis. * p< 0.05 ** ps 
0.01 *** p < 0.001 
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Annex Table 3 Overall treatment effect on expectations of success and the perceived utility of 
STEM subjects for Very High-Achieving Girls 


| know | can | beli I can apply pds 
learn the sal what | learn pees “an 
Coefficient Helle successful LTO (err be 
N cover in Sig. | . Sig. science Sig. | . Sig. 
ame in maths or important 
maths or P class to 
science ey x real life ase 
(Coef.) ; (Coef.) (Coef.) 
a -0.06 0.01 0.19 ws 0.05 
treatment 
(0.06) (0.06) (0.06) (0.06) 
Student fixed 
effects yes yes yes yes 
R2 0.79 0.82 0.81 0.84 
Mean in pre- 
treatment 4.84 4.64 4.01 4.78 
N 286 284 284 285 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
very high-achieving girls with complete data (those who responded to the survey questions before exercise 
1 and then again after exercise 3). Standard errors are reported in parenthesis. * p < 0.05 ** p< 0.01 *** p 
< 0.001 


Annex Table 4 Overall treatment effect on self-reported discussions with parents/guardians for 
Very High-Achieving Girls 


Fee ree seks aoe l have talked to a parent or 
sett | eee grime | oo | pe seer | a 
oe é : : : 
nye levelichelcesiniie : Beene in the last month : 
last month (Coef.) 
(Coef.) 
After treatment 0.00 -0.02 
(0.03) (0.03) 
Student fixed 
effects yes yes 
R2 0.76 0.75 
Mean in pre- 
treatment 0.65 0.43 
N 284 285 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 
very high-achieving girls with complete data (those who responded to the survey questions before exercise 
1 and then again after exercise 3). Standard errors are reported in parenthesis. * p < 0.05 ** p< 0.01 *** p 
< 0.001 
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Annex Table 5 Relative impact of different treatments on expectations of success and the 


perceived utility of STEM subjects for Very High-Achieving Girls 


| know I can ‘ “llth: 
learn the | believe | I can apply maths or 
ene material we can a what Pat ras will 
oefficien : : successfu : in maths or : e : 
Name maonaleuh ele: in maths or ele: science Sle: important Sig. 
maths or : 
science science class to real to my 
(Coef.) (Coef.) life (Coef.) future 
(Coef.) 
After treatment -0.03 0.03 0.21 i -0.02 
(0.07) (0.07) (0.08) (0.07) 
Pareniiocused 0.44 0.53 1.97 -1.46 * 
intervention 
(0.69) (0.66) (0.75) (0.70) 
After treatment 
ee 0.11 -0.05 0.06 0.08 
ocused 
intervention 
(0.12) (0.12) (0.13) (0.12) 
Student fixed 
effects yes yes yes yes 
R2 0.79 0.82 0.81 0.84 
Mean in pre- 
treatment 4.84 4.64 4.01 4.78 
N 286 284 284 285 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 


very high-achieving girls with complete data (those who responded to the survey questions before exercise 


1 and then again after exercise 3). Standard errors are reported in parenthesis. * p < 0.05 ** p < 0.01 *** p 


< 0.001 
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Annex Table 6 Relative impact of different treatments on self-reported discussions with 
parents/guardians for Very High-Achieving Girls 


Coefficient Name 


| have had at least one 
conversation with a parent 
or guardian about my A 
level choices in the last 
month (Coef.) 


Sig. 


| have talked to a parent or 
guardian at least once about the 
importance of maths or science 
in the last month (Coef.) 


Sig. 


After treatment -0.03 -0.02 

(0.03) (0.04) 
Parent-focused 0.96 a 0.00 
intervention 

(0.33) (0.35) 
After treatment * 
Parent-focused 0.08 0.00 
intervention 

(0.06) (0.06) 
Student fixed effects yes yes 
R2 0.76 0.75 
Mean in pre- 
treatment 0.65 0.43 
N 284 285 


Data collected through the Behavioural Insights Team's bespoke online platform. Sample restricted to the 


very high-achieving girls with complete data (those who responded to the survey questions before exercise 


1 and then again after exercise 3). Standard errors are reported in parenthesis. * p < 0.05 ** p< 0.01 *** p 


< 0.00 
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